What is transfer learning?
Transfer learning reuses a trained model for training another similar type of model with similar neural architecture. Transfer learning results in getting high accuracy at a much smaller training data set and lower computational cost. It can be used for training complex ML models since most real-world problems typically do not have labeled data points to train such complex issues.
In transfer learning, a machine exploits another model’s knowledge for solving a particular problem and uses that generalized information for training other ML models at a much higher efficiency.
For instance, training a classifier to predict whether an image contains an animal or car could use the knowledge it gained during training to differentiate between vehicles and animals.
Why use it?
Transfer learning has several benefits. Using transfer learning results in saving training time; moreover, it improves the performance of neural networks (in most cases) and not needing a lot of data. Generally, a lot of data is needed to train a neural network from scratch, but access to that data is usually difficult and costly; this is where transfer learning comes into light. With the help of transfer learning, a reliable and accurate machine learning model is often built with a comparatively smaller training data set. The model is already pre-trained, which makes it valuable. In natural language processing, expert knowledge is precious for creating large labeled datasets. Additionally, training time is reduced as it takes days or even weeks to train a deep neural network from scratch on a complicated task.
Popular pre-trained models
Some popular pre-trained models are available to use. These models are trained on millions of images with high variance. They are using these models for transfer learning results in higher accuracy as compared to the traditional approach.
- Inception-v3 model was trained for the ImageNet, which is currently the Large Visual Recognition Challenge. In this challenge, teams had to classify images into 1,000 classes like “horse,” “car,” and “dishwasher.”
- RESNET50 is also a prevalent model with 50 layers, which performs extremely well in classifying tasks. For deeper networks like ResNet50, ResNet152, etc., a bottleneck design is used. Each residual function present are three layers stacked one over the other. The three layers present are 1×1, 3×3, 1×1 convolutions. The 1×1 convolution layers present in it are used for reducing the dimension and then restoring the dimensions. The 3×3 layer is left as a bottleneck having smaller sizes.
- VGG-16 is a simpler architecture model since it’s not using many hyperparameters. It always uses 3 x 3 filters with a stride of 1 in the convolution layer and uses SAME padding in pooling layers 2 x 2 with a stride of 2. The winner of ILSVRC 2014 and the GoogLeNet architecture is also referred to as the Inception Module.