# Model‑Agnostic Meta‑Learning (MAML)

Model-Agnostic Meta-Learning (MAML) is an optimization-based algorithm for few-shot meta-learning, enabling models to adapt quickly to new tasks with minimal gradient steps by learning an initialization that is broadly applicable across task distributions.

Model-Agnostic Meta-Learning (MAML) is an algorithm for few-shot meta-learning introduced in 2017 by Chelsea Finn, Pieter Abbeel, and Sergey Levine at the University of California, Berkeley. It is designed to train a model on a distribution of tasks such that it can adapt to a new task with only a small number of gradient updates and a small amount of training data. The term 'model-agnostic' refers to the algorithm's compatibility with any model trained via gradient descent, including [neural networks](https://www.wikiprompt.org/wiki/neural-network) and other differentiable architectures, without requiring task-specific architectural modifications.

MAML operates on the principle of learning an initialization of model parameters that serves as a good starting point for rapid adaptation. Rather than learning a single model that performs well on all tasks, MAML learns a meta-initialization from which a few gradient steps on a new task's data yield a model specialized to that task. This contrasts with standard [machine learning](https://www.wikiprompt.org/wiki/machine-learning) approaches that optimize for performance on a fixed dataset, and with transfer learning methods that typically require more extensive fine-tuning.

## Algorithm Overview

The MAML algorithm involves a two-level optimization process: an inner loop and an outer loop. In the inner loop, for each task in a batch sampled from a task distribution, the model performs one or more gradient descent steps on the task's training set (support set), producing task-specific parameters. In the outer loop, the meta-objective is to minimize the loss of these adapted parameters on the task's validation set (query set), with gradients computed through the inner-loop updates. This bi-level optimization allows the model to learn parameters that are sensitive to task-specific changes, meaning small perturbations in the parameter space lead to significant improvements in task performance.

The meta-objective can be expressed as minimizing the sum of losses over tasks after a fixed number of inner gradient steps. For a model with parameters θ, a task T with loss function L_T, and k inner steps, the meta-loss is Σ_T L_T(θ - α ∇L_T(θ)) for k=1, where α is the inner learning rate. The outer update uses a meta-learning rate β to adjust θ. This formulation requires computing second-order derivatives, though first-order approximations (FOMAML) are often used to reduce computational cost, as empirically they perform nearly as well.

## Applications in Few-Shot Learning

MAML has been widely applied to few-shot classification, regression, and reinforcement learning problems. In few-shot image classification, such as on the Omniglot and Mini-ImageNet benchmarks, MAML achieves competitive accuracy with only one to five examples per class. For example, on Mini-ImageNet, MAML with a convolutional neural network backbone reaches approximately 48.7% accuracy on 5-way 1-shot tasks and 63.1% on 5-way 5-shot tasks, as reported in the original paper. These results demonstrated that learning to learn can match or exceed prior metric-based meta-learning methods.

In reinforcement learning, MAML has been used to enable agents to adapt to new environments or reward functions quickly. The algorithm's model-agnostic nature allows it to be applied to policy gradient methods, enabling rapid adaptation in simulated robotic control tasks, such as a two-link arm reaching and a locomotion task, where the agent learns to adjust its policy with a few gradient updates based on new dynamics.

## Extensions and Variants

Several extensions have been proposed to address MAML's limitations, particularly its computational cost and sensitivity to task distribution. Reptile, introduced by Alex Nichol, Joshua Achiam, and John Schulman in 2018, simplifies MAML by performing multiple inner gradient steps and then moving the meta-parameters toward the final adapted parameters, avoiding second-order derivatives entirely. Another variant, Meta-SGD, learns both the initialization and the learning rate per parameter, providing more flexibility. Probabilistic MAML (PMAML) models the parameter initialization as a distribution to handle uncertainty in task adaptation.

MAML has also been combined with [Transformer](https://www.wikiprompt.org/wiki/transformer) architectures and [large language models](https://www.wikiprompt.org/wiki/large-language-model) for few-shot prompting and in-context learning, though these models often rely on their own meta-learning capabilities. The algorithm's principles have influenced research in [artificial intelligence](https://www.wikiprompt.org/wiki/artificial-intelligence) more broadly, including work at institutions like [Stanford AI Lab](https://www.wikiprompt.org/wiki/stanford-ai-lab) and [Berkeley AI Research](https://www.wikiprompt.org/wiki/berkeley-ai-research).

## Theoretical and Practical Considerations

The theoretical underpinnings of MAML relate to the idea of learning a good prior over parameter space. The algorithm implicitly assumes that tasks in the distribution share some common structure, allowing a single initialization to be effective. However, MAML can suffer from issues such as meta-overfitting when the task distribution is narrow, and it may require careful tuning of hyperparameters like inner and outer learning rates. The computational cost of backpropagating through the inner loop can be high, especially with many inner steps, though first-order approximations mitigate this.

Practically, MAML has been implemented in popular [deep learning](https://www.wikiprompt.org/wiki/deep-learning) frameworks and has become a standard baseline in meta-learning research. Its model-agnostic property means it can be applied to any differentiable model, including those used in [generative AI](https://www.wikiprompt.org/wiki/generative-ai) and other domains. As of the early 2020s, MAML remains a foundational algorithm in few-shot learning, with ongoing research exploring its connections to Bayesian inference and continual learning.

## Impact and Legacy

MAML has had a significant impact on the field of meta-learning, inspiring a large body of follow-up work and becoming a reference point for comparing new algorithms. Its introduction helped popularize the idea of learning to learn through gradient-based optimization, distinct from memory-based or metric-based approaches. The algorithm's success has led to its adoption in various real-world applications, including personalized [machine learning](https://www.wikiprompt.org/wiki/machine-learning) models in healthcare and robotics, where rapid adaptation to new conditions is critical. Researchers at institutions such as [Carnegie Mellon University](https://www.wikiprompt.org/wiki/carnegie-mellon-university) and [MIT CSAIL](https://www.wikiprompt.org/wiki/mit-csail) have built upon MAML, and it continues to be a topic of active investigation in the broader context of [AI](https://www.wikiprompt.org/wiki/artificial-intelligence) research.

---
Source: https://www.wikiprompt.org/wiki/maml
License: CC BY-SA 4.0 (https://creativecommons.org/licenses/by-sa/4.0/)
Last updated: 2026-09-07T02:33:41.796971+00:00