Introduction to Transfer Learning

Source:https://www.analyticssteps.com/backend/media/thumbnail/1967565/9315476_1592890541_transfer.jpg

Introduction to Transfer Learning

Transfer learning is a machine learning technique in which a model trained on one task is repurposed on another related activity. So basically it is the reuse of a previously learned model on a new problem known as transfer learning. Transfer learning is an optimization technique that allows rapid improved performance when modeling the second task.Nevertheless, transfer learning is famous in deep learning given the enormous resources required to train deep learning models or the large and challenging datasets on which deep learning models are trained.It is important to highlight that transfer learning is only effective in deep learning when the model features learnt from the first assignment are generic.

Transfer Learning Approaches 

Two common approaches are as follows:

1. Develop Model Approach
2. Pre-trained Model Approach

Develop Model Approach

1. Pickup the source Task. You must select a much related predictive modeling problem with an abundance of data where there is some relationship in the input data, output data, or concepts learned during the mapping from input to output data.
2. Develop Source Model. You must develop a generalizable model for this first task. The model must be better than a basic model to make sure that some feature learning has been performed.
3. Reuse Model. The model which is fitted on the source task can then be used as the starting point for a model on the second task of interest. This might  involve using all the parts of the model or some parts of the model, depending on the selected modeling technique used.
4. Model Tuning. The model may need to be modified or updated based on the input-output pair data available for the job of interest.

Pre-trained Model Approach

1. Pickup Source Model. A pre-trained source model is chosen from the available models. Many research institutions released models on large and challenging datasets that may be included in the pool of candidate models from which to choose from.
2. Reuse Model. The model which is pre-trained  can then be used as the starting point for a model on the second task of interest. Depending on the modelling technique employed, this could entail employing the entire model or just a subset.
3. Tune the Model. The model may need to be modified or updated based on the input-output pair data available for the job of interest.

This second type of transfer learning method is most famous in the field of deep learning.

Transfer Learning Usage

Transfer learning is an optimization for saving time or getting better performance. Also there is no guarantee that there will be a benefit in using transfer learning in the domain until after the model has been developed and evaluated first.

The three potential advantages to look for when applying transfer learning:

Higher start. The source model's initial skill (prior to refinement) is higher than it would otherwise be.
Slope is higher. The rate of skill improvement during source model training is higher than it would otherwise be.
Increased asymptote. The trained model has a higher converged skill than it would have otherwise.
A great transfer learning application should provide you with all three benefits.

It is a useful strategy to try if you can discover a relevant task with some data and you have the resources to construct a model for that work and reuse it on your own problem, or a pre-trained model is available that you can use as a starting point for your own model.

On some of the problems where you may not have very much data, transfer learning can enable you to develop skillful models that you are simply unable to develop in the absence of transfer learning.

The choice of source data or source model is an open problem and may require domain expertise that is developed via experience.

Conclusion

Hence Transfer learning is one of the important strategies in the deep learning field especially when we have very little data to train.