LLaMA 2 is a family of large language models (LLMs) released by Meta AI on July 18, 2023, in partnership with Microsoft. It represents the second generation of the LLaMA (Large Language Model Meta AI) series, following the initial release in February 2023. The models were made available in three parameter sizes - 7 billion, 13 billion, and 70 billion - and included both foundation models and versions fine-tuned for chat applications. A key departure from its predecessor was the licensing: all LLaMA 2 models were released with weights and permitted many commercial use cases, though the license's acceptable use policy meant it was not considered open source by the Open Source Initiative.
The release marked a significant shift in Meta's approach to AI model distribution. While the first LLaMA model was restricted to academic researchers on a case-by-case basis, LLaMA 2 was made accessible to a broader audience, including commercial entities. This move was part of a wider trend in the industry toward more open model releases, though it also sparked debate about the definition of open source in the context of AI. The model architecture remained largely unchanged from LLaMA 1, but training data was increased by 40%, enhancing performance across various benchmarks.
Background and Context
The development of LLaMA 2 occurred against a backdrop of rapid advancement in large language models. Following the success of models like GPT-3 and the surprise popularity of ChatGPT, there was intense competition among tech companies to produce capable LLMs. Meta's Chief AI scientist Yann LeCun had publicly stated that large language models were best suited for aiding with writing tasks, positioning Meta's approach within the broader discourse on AI capabilities.
The first LLaMA model, announced on February 24, 2023, had already demonstrated that smaller models could compete with much larger ones. The 13B parameter version outperformed GPT-3 (175B parameters) on most NLP benchmarks, while the 65B model was competitive with state-of-the-art systems like PaLM and Chinchilla. This efficiency was partly due to training on publicly available data and the use of advanced techniques in deep learning and neural network design.
Model Architecture and Training
LLaMA 2's architecture was based on the Transformer (architecture) framework, which had become the standard for large language models. The models were trained using multi-head attention mechanisms and positional encoding, consistent with the design of LLaMA 1. The training process involved scaling up the dataset by 40% compared to the first version, allowing the models to learn from a more extensive corpus of text.
The 70B parameter model was the largest in the family, designed for high-performance applications, while the 7B and 13B versions offered more accessible options for researchers and developers with limited computational resources. All models were released as both foundation models and instruction-tuned versions, the latter fine-tuned for chat and conversational tasks. This dual release was a first for the LLaMA series, which had initially been exclusively foundation models.
Licensing and Commercial Availability
A defining feature of LLaMA 2 was its licensing model. Unlike LLaMA 1, which was restricted to academic researchers under a non-commercial license, LLaMA 2 was released under a license that permitted commercial use, subject to an acceptable use policy. This policy prohibited certain applications, such as those involving illegal activities or harm, but allowed for a wide range of legitimate uses.
Meta's characterization of LLaMA 2 as "open source" was disputed by the Open Source Initiative, which maintains The Open Source Definition. Critics argued that the acceptable use policy and other restrictions meant the license did not meet the criteria for true open source. This debate highlighted ongoing tensions in the AI community about what constitutes openness in model releases.
The commercial availability of LLaMA 2 was significant for businesses and startups, enabling them to integrate the model into products without the need for expensive licensing fees from proprietary providers. This was seen as a counterweight to the dominance of closed models from companies like OpenAI and Anthropic.
Impact and Reception
The release of LLaMA 2 was met with considerable interest in the AI community. Its performance, particularly the 70B model, was competitive with other leading models of the time, and the permissive license made it an attractive option for both research and commercial deployment. The availability of multiple sizes allowed users to choose a model that matched their hardware capabilities, from edge devices to large-scale servers.
Following LLaMA 2, Meta continued to develop the series, releasing Code Llama in August 2023 for code-specific tasks, and later LLaMA 3 in April 2024 with improved performance and larger training datasets. The LLaMA family became a cornerstone of Meta's AI strategy, culminating in the development of Meta AI, an assistant built on LLaMA, and the eventual release of LLaMA 4 in April 2025. In April 2026, Meta Superintelligence Labs introduced Muse Spark as a replacement for LLaMA, marking the next evolution of Meta's AI models.
Technical Details and Variants
The LLaMA 2 models were trained using techniques common in modern LLM development, including Adam optimizer and learning rate schedules. The training process also employed gradient clipping and layer normalization to stabilize training and improve convergence. These methods were standard in the field, reflecting the state of the art in machine learning at the time.
Code Llama, a specialized variant of LLaMA 2, was fine-tuned on code-specific datasets. The 7B, 13B, and 34B versions were released on August 24, 2023, with a 70B version following on January 29, 2024. Training involved an additional 500 billion tokens of code data, followed by 20 billion tokens of long-context data, creating foundation models for general code generation. Further fine-tuning on 5 billion instruction-following tokens produced the instruct versions, and a separate Python-focused model was trained on 100 billion tokens of Python code.
These technical developments positioned LLaMA 2 as a versatile platform for various applications, from natural language processing to software development. The release also spurred innovation in the broader ecosystem, with companies like Amazon Web Services and Azure offering LLaMA 2 on their cloud platforms, making it accessible to a wider range of users.