Llama 3 is a family of large language models developed by Meta AI and released on April 18, 2024. It is part of the Llama series, which began in February 2023, and represents a significant advancement in open-weight model performance. The initial release included two model sizes: 8 billion (8B) and 70 billion (70B) parameters, with both foundation and instruction-tuned versions. Llama 3 models were pre-trained on approximately 15 trillion tokens of text from publicly available sources, with instruction-tuned variants further fine-tuned on public instruction datasets and over 10 million human-annotated examples.
Llama 3 builds on the architectural principles of its predecessors but introduces notable improvements in training data scale and performance. According to Meta AI's testing in April 2024, the 70B model outperformed competing models such as Gemini Pro 1.5 and Claude 3 Sonnet on most benchmarks. The release also marked the rollout of Meta AI, an AI assistant built on Llama, available via a dedicated website and integrated into Facebook and WhatsApp.
Background
The development of Llama 3 occurred against a backdrop of rapid progress in generative AI, particularly following the release of ChatGPT in late 2022. The success of ChatGPT intensified interest in large language models, prompting companies to invest heavily in scaling and refining these systems. Meta's Chief AI scientist, Yann LeCun, had previously expressed that large language models are best suited for aiding with writing, a perspective that influenced the design of Llama models.
Llama 3 also reflects a shift in Meta's approach to model distribution. While the original Llama was restricted to academic researchers under a non-commercial license, subsequent versions, including Llama 3, were made accessible to a broader audience under licenses permitting some commercial use. This openness has spurred innovation but also raised debates about the definition of "open source" in the AI community.
Architecture and Training
Llama 3 models use a standard Transformer (architecture) architecture, similar to earlier Llama versions, but with enhancements in training methodology. The models were trained on a dataset of 15 trillion tokens, a substantial increase from the 1.4 trillion tokens used for Llama 2. This scale allowed the models to achieve better performance across a wide range of tasks.
A key finding from Llama 3 training was related to scaling laws. The models empirically demonstrated that performance continues to improve log-linearly even when training data exceeds the "Chinchilla-optimal" amount. For instance, the Chinchilla-optimal dataset for the 8B model is 200 billion tokens, but performance kept scaling up to the 15 trillion tokens used, which is 75 times larger. This observation has implications for future model training strategies.
During an interview with Dwarkesh Patel, Meta CEO Mark Zuckerberg noted that the 8B version of Llama 3 was nearly as powerful as the largest Llama 2 model. He also mentioned that the 70B model was still learning at the end of its 15 trillion token training, but the decision was made to stop training to allocate GPU resources elsewhere.
Performance and Benchmarks
Llama 3 models achieved state-of-the-art results on many natural language processing benchmarks at the time of release. The 70B model, in particular, outperformed several proprietary models, including Gemini Pro 1.5 and Claude 3 Sonnet, on tasks such as reasoning, coding, and general knowledge. The 8B model, despite its smaller size, delivered competitive performance, making it suitable for deployment on less powerful hardware.
Meta AI's evaluation showed that Llama 3 70B excelled in areas like mathematical reasoning and code generation, while the 8B model offered a strong balance of efficiency and capability. These results positioned Llama 3 as a leading open-weight model, challenging the dominance of closed-source systems.
Availability and Ecosystem
Llama 3 was released under a license that permits commercial use, with an acceptable use policy that restricts certain applications. The models are available for download from Meta's website and through various cloud platforms, including Amazon Web Services, Azure, and Google Cloud. This accessibility has facilitated widespread adoption in research and industry.
The release of Llama 3 also coincided with the launch of Meta AI, an assistant that leverages the model's capabilities. Meta AI is integrated into Facebook, WhatsApp, and a dedicated website, providing users with conversational AI services. The ecosystem around Llama 3 includes tools for fine-tuning, quantization, and deployment, enabling developers to customize the models for specific use cases.
Legacy and Future
Llama 3 marked a milestone in the evolution of open-weight large language models, demonstrating that such models could rival proprietary counterparts. Its success influenced subsequent developments, including Llama 3.1 released on July 23, 2024, which introduced further improvements. The Llama series continued with Llama 4 in April 2025, and in April 2026, Meta Superintelligence Labs released Muse Spark as a replacement for Llama.
The impact of Llama 3 extends beyond technical achievements, as it contributed to discussions about AI safety, accessibility, and the democratization of advanced AI technologies. Its open-weight nature has enabled researchers and developers worldwide to experiment and build upon the model, fostering a vibrant ecosystem of innovation.