Transfer learning (TL) is a technique in machine learning in which knowledge learned from a task is re-used in order to boost performance on a related task. For example, for image classification, knowledge gained while learning to recognize cars could be applied when trying to recognize trucks. This topic is related to the psychological literature on transfer of learning, although practical ties between the two fields are limited. Reusing or transferring information from previously learned tasks to new tasks has the potential to significantly improve learning efficiency. Since transfer learning makes use of training with multiple objective functions it is related to cost-sensitive machine learning and multi-objective optimization.
In modern practice, the dominant form of transfer learning is pretraining a large neural network on a broad, data-rich source task, then fine-tuning it on a narrower target task with limited data. This approach underpins many successes in deep learning, especially in natural language processing (as seen with large language models) and computer vision. The term 'transfer learning details' often refers to the practical and theoretical considerations involved in this fine-tuning process.
History
In 1976, Bozinovski and Fulgosi published a paper addressing transfer learning in neural network training. The paper gave a mathematical and geometrical model of the topic. In 1981, a report considered the application of transfer learning to a dataset of images representing letters of computer terminals, experimentally demonstrating positive and negative transfer learning. Negative transfer occurs when the source knowledge harms target performance.
In 1992, Lorien Pratt formulated the discriminability-based transfer (DBT) algorithm)Skip_scanme. By 1998, the field had advanced to include multi-task learning, along with more formal theoretical foundations. Influential publications on transfer learning include the book Learning to Learn in 1998, a 2009 survey, and a 2019 survey.
Andrew Ng said in his NIPS 2016 tutorial that transfer learning would become the next driver of machine learning commercial success after supervised learning. In the 2020 paper 'Rethinking Pre-Training and Self-Training', Zoph et al. reported that pre-training can hurt accuracy in some settings and advocated self-training instead, highlighting the nuanced trade-offs of transfer learning.
Definition
The definition of transfer learning is given in terms of domains and tasks. A domain D consists of: a feature space X and a marginal probability distribution P(X), where X = {x_1, ..., x_n} ∈ X. Given a specific domain D = {X, P(X)}, a task consists of two components: a label space Y and an objective predictive function f: X → Y. The function f predicts the label f(x) of a new instance x. This task, denoted T = {Y, f(x)}, is learned from training data consisting of pairs {x_i, y_i}.
Given a source domain D_S and learning task T_S, and a target domain D_T and learning task T_T, where D_S ≠D_T or T_S ≠T_T, transfer learning aims to help improve the learning of the target predictive function f_T(·) in D_T using the knowledge in D_S and T_S. The key assumption is that the source and target tasks are related enough that shared knowledge can be transferred.
Mechanism of Fine-Tuning
The most common implementation of transfer learning in deep learning is fine-tuning. A model is first pretrained on a large, general dataset (the source task). Then, its weights are used as initialization for training on the target task. The pretrained network's lower layers, which typically learn generic features, are often frozen or adapted with a low learning rate to preserve the learned representations, while higher layers are retrained on the target data.
Fine-tuning can be applied to the entire network or only to a subset of layers. When the target dataset is small, practitioners often freeze most layers to avoid overfitting and only update the final classification head. For larger target datasets, more aggressive fine-tuning is possible. Techniques like dropout, batch normalization, and layer normalization are commonly used during fine-tuning to regularize the model.
Types of Transfer Learning
Transfer learning can be categorized based on the level of agreement between source and target domains and tasks. Inductive transfer learning occurs when the source and target tasks differ, even if the domains are the same. Transductive transfer learning occurs when the domains differ but the tasks are the same. Unsupervised transfer learning applies when both tasks and domains differ but labeled data is unavailable.
In practice, most deep learning applications use inductive transfer learning with identical feature spaces - for example, pretraining a transformer on generic text and fine-tuning it on sentiment analysis. Multi-task learning, where a model is trained on multiple objectives simultaneously, is closely related and shares mathematical foundations with transfer learning.
Applications
Transfer learning is endemic across AI applications. In computer vision, models pretrained on ImageNet have been fine-tuned for medical image analysis, object detection, and autonomous driving. In natural language processing, OpenAI and Google DeepMind have released large pretrained models that researchers fine-tune for specific tasks such as question answering or translation.
Specific examples include BERT for text understanding, which was pretrained on masked language modeling and fine-tuned for sentiment classification or named entity recognition. In speech recognition, models pretrained on general audio are adapted to acoustic environments. In reinforcement learning, transfer learning can speed up learning on new environments by reusing policies from prior tasks.
Theoretical Considerations
The effectiveness of transfer learning depends on the similarity between source and target distributions. A key theoretical result is that the generalization error on the target task is bounded by the error on the source task plus a term that measures the divergence between the two domains. This relationship guides practical decisions about when to apply transfer learning.
Negative transfer can occur when the source knowledge is misleading for the target task. For example, pretraining on image classification with many classes may not help with a target task that has a very different label distribution. Domain adaptation techniques, such as adversarial training, aim to minimize the domain shift. Generally, recent research emphasizes that transfer learning is not a silver bullet; careful evaluation is needed.
Optimization Techniques
Fine-tuning large pretrained models requires specialized optimization. The Adam optimizer and its variants are standard choices, often with a lower initial learning rate than for training from scratch. Gradient clipping is frequently used to prevent explosion during fine-tuning. Curriculum learning can also be applied, where the target dataset is presented in increasing difficulty.
Another approach is to use a smaller learning rate for the pretrained layers and a higher one for the newly initialized layers. Additionally, data augmentation techniques, such as random cropping or token masking, can help the model generalize on small target datasets. Some methods use model pruning after fine-tuning to reduce model size without significant accuracy loss.
Relationship to Large Language Models
The rise of large language models has made transfer learning more critical than ever. Models like GPT-3 and Claude are pretrained on vast corpora and then adapted for downstream tasks through fine-tuning or prompting. In some cases, transfer learning is performed at scale via instruction tuning, where a model is fine-tuned on many tasks simultaneously to improve generalization.
Reinforcement learning from human feedback (RLHF) is a form of transfer learning where a pretrained model is fine-tuned using human preferences as a reward signal. This approach has been instrumental in aligning models with human values. Research from Anthropic and OpenAI illustrates how transfer learning interfaces with safety and alignment.
Challenges and Future Directions
A major challenge is the computational cost of pretraining, which often requires large clusters of specialized hardware such as AWS Trainium or Google Cloud TPUs. There is active research on efficient transfer learning, including parameter-efficient methods like adapters or LoRA that update only a small fraction of weights.
Another concern is forgetting - when fine-tuning on a new task, the model may lose knowledge from the source task. Continual learning approaches aim to mitigate this. As of the early 2020s, transfer learning remains an active research area, with nuanced findings about when pre-training helps and when it does not, as highlighted by Zoph et al. Future work may focus on theoretical guarantees and better methods for avoiding negative transfer.