Meta Llama is a family of large language models (LLMs) developed by Meta AI, first released in February 2023. The name "Llama" serves as a backronym for "Large Language Model Meta AI." The models are designed to be accessible for research and commercial use, with varying parameter sizes ranging from 1 billion to over 2 trillion. Initially, Llama was a foundation model, but starting with Llama 2, Meta AI also released instruction fine-tuned versions. The latest version, Llama 4, was released in April 2025, and in April 2026, Meta Superintelligence Labs introduced Muse Spark as a replacement for Llama.
Background
The development of Meta Llama occurred against the backdrop of rapid advancements in large language models. After the release of models like GPT-3, research focused on up-scaling, which sometimes led to significant gains in emergent capabilities. The launch of ChatGPT and its unexpected success heightened public and industry attention on LLMs. In response to ChatGPT, Meta's Chief AI scientist Yann LeCun commented that large language models are particularly useful for aiding with writing tasks, reflecting a cautious but engaged stance from Meta.
Initial Release and Leak
The first version of Llama, stylized as LLaMA and sometimes called Llama 1, was announced on February 24, 2023, through a blog post and a paper detailing its training, architecture, and performance. The inference code was released under the open-source GPLv3 license, but access to the model weights was restricted to academic researchers and affiliated organizations, granted on a case-by-case basis. The model was trained on publicly available data and came in multiple sizes to accommodate different hardware capabilities. Meta reported that the 13B parameter model outperformed the much larger GPT-3 (175B parameters) on most NLP benchmarks, and the largest 65B model was competitive with state-of-the-art models like PaLM and Chinchilla.
On March 3, 2023, a torrent containing Llama's weights was leaked on 4chan and spread through AI communities. Meta filed takedown requests against HuggingFace repositories and a GitHub script, citing unauthorized distribution and copyright infringement. Reactions were mixed: some feared misuse, while others celebrated the accessibility, comparing it to the open distribution of Stable Diffusion, which spurred rapid innovation.
Llama 2
On July 18, 2023, in partnership with Microsoft, Meta announced Llama 2, available in 7B, 13B, and 70B parameter sizes. The architecture was largely unchanged from Llama 1, but training data was increased by 40%. Llama 2 included both foundation and chat fine-tuned models, with weights released for commercial use under a license that imposed an acceptable use policy, leading to disputes over whether it qualifies as open source. The Open Source Initiative argued that the license restrictions prevent it from being truly open source.
Code Llama, a fine-tune of Llama 2 for code generation, was released on August 24, 2023, in 7B, 13B, and 34B versions, with a 70B version following on January 29, 2024. The training involved additional 500B tokens of code data and 20B tokens of long-context data, with further fine-tuning for instruction following and Python-specific models.
Llama 3
On April 18, 2024, Meta released Llama 3 with 8B and 70B parameter sizes. These models were pre-trained on approximately 15 trillion tokens from publicly available sources, with instruct models fine-tuned on public instruction datasets and over 10 million human-annotated examples. Meta's benchmarks showed Llama 3 70B outperforming Gemini Pro 1.5 and Claude 3 Sonnet on most tests. Meta announced plans for multilingual and multimodal capabilities, improved coding and reasoning, and a larger context window.
Llama 3 demonstrated that performance continues to scale log-linearly even beyond the Chinchilla-optimal training data amount. For instance, the Chinchilla-optimal dataset for the 8B model is 200 billion tokens, but performance improved up to 15 trillion tokens. Mark Zuckerberg noted that the 8B version was nearly as powerful as the largest Llama 2, and the 70B model was still learning at the end of training, which was stopped to allocate GPU resources elsewhere.
Llama 3.1 was released on July 23, 2024, with further improvements and expanded context lengths.
Llama 4 and Successor
Llama 4 was released in April 2025, continuing the evolution of the model family with enhanced capabilities. In April 2026, Meta Superintelligence Labs released Muse Spark as a replacement for Llama, marking a transition to a new generation of models.
Applications and Ecosystem
Meta Llama models have been integrated into various products and services. Alongside Llama 3, Meta launched Meta AI, an AI assistant built on Llama, available on a dedicated website and platforms like Facebook and WhatsApp. The models are also used by third-party developers and cloud providers, including Amazon Web Services, Microsoft Azure, and Google Cloud, to offer LLM capabilities to enterprises. The open-weight nature of Llama has fostered a vibrant ecosystem of fine-tuned variants and tools, similar to the community around generative AI models.
Reception and Impact
The release of Llama models has had a significant impact on the AI landscape. The leak of Llama 1 democratized access to powerful LLMs, enabling research and experimentation outside large corporations. Llama 2 and subsequent versions provided a commercially viable alternative to proprietary models like those from OpenAI and Anthropic. However, the licensing terms have sparked debates about the definition of open source, with critics noting that the acceptable use policy restricts certain applications. Despite this, Llama has become a benchmark for open-weight models, influencing the development of other open-source efforts in artificial intelligence.
Technical Architecture
Llama models are based on the Transformer architecture, utilizing multi-head attention and positional encoding. They employ techniques such as layer normalization and residual connections to stabilize training. The models are trained using SGD variants like Adam, with learning rate schedules and gradient clipping. For inference, top-k sampling and top-p sampling are commonly used, along with temperature scaling. The scaling laws observed in Llama 3 have informed research on optimal training data sizes, contributing to the broader understanding of deep learning and neural networks.
Future Directions
With the introduction of Muse Spark in 2026, Meta Superintelligence Labs aims to push the boundaries of AI capabilities, potentially incorporating advances in machine learning and generative AI. The transition from Llama to Muse Spark suggests a strategic shift toward more powerful and efficient models, likely building on the lessons learned from Llama's development. As the field evolves, Meta's contributions continue to shape the landscape of open-weight AI models, balancing innovation with considerations of safety and accessibility.