There are several ways to prevent overfitting, two of which are:
\nA single model can be used to simulate having a large number of different network architectures by randomly dropping out nodes during training. This is called dropout and offers a very computationally cheap and remarkably effective regularization method to reduce overfitting and improve generalization error in deep neural networks of all kinds. ref\n
The following figure shows a network with dropout.\n
![\"dropout\"](\"https://miro.medium.com/max/1400/1*iWQzxhVlvadk6VAJjsgXgg.png\")
\nimage data augmentation is a technique that can be used to artificially expand the size of a training dataset by creating modified versions of images in the dataset.
Training deep learning neural network models on more data can result in more skillful models, and the augmentation techniques can create variations of the images that can improve the ability of the fit models to generalize what they have learned to new images. ref
Some image augmentation techniques include:
\nThe following figure shows the augmentation of a cat image.
\n![\"augmentation\"](\"https://929687.smushcdn.com/2633864/wp-content/uploads/2021/05/tf_data_data_aug_sequential.png?lossy=1&strip=1&webp=1\")
\nTransfer learning is a machine learning technique where a model trained on one task is re-purposed on a second related task. ref
We can use 2 below strategies for doing transfer learning:
Instead of using the model end-to-end as in the previous example, we can treat the pre-trained neural network as a feature extractor by discarding the last fully-connected output layer. This approach allows us to directly apply new dataset to solve an entirely different problem. ref
\nUnfreeze a few of the top layers of a frozen model base and jointly train both the newly-added classifier layers and the last layers of the base model. This allows us to \"fine-tune\" the higher-order feature representations in the base model in order to make them more relevant for the specific task. ref
\n