LLaMA 3 is a family of large language models released by Meta AI on April 18, 2024. It is the third generation of the Llama series, succeeding Llama 2, and includes two initial model sizes: 8 billion and 70 billion parameters. The models were pre-trained on approximately 15 trillion tokens of text from publicly available sources, with instruction-tuned versions fine-tuned on public instruction datasets and over 10 million human-annotated examples. LLaMA 3 demonstrated competitive performance against proprietary models like Gemini Pro 1.5 and Claude 3 Sonnet on many benchmarks, and Meta announced plans to make it multilingual, multimodal, and better at coding and reasoning.
Background
The development of LLaMA 3 occurred amid rapid advances in generative AI, following the success of OpenAI's ChatGPT and the release of earlier Llama models. Meta's Chief AI scientist Yann LeCun had stated that large language models are best suited for aiding with writing, reflecting Meta's focus on practical applications. The Llama series began in February 2023 with the original LLaMA, which was initially available only to researchers under a non-commercial license. After an unauthorized leak of the weights via BitTorrent, subsequent versions, including Llama 2, were made more accessible with commercial use permitted.
Model Architecture and Training
LLaMA 3 retains the transformer architecture used in previous Llama models but with improvements in training data scale and quality. The models were trained on 15 trillion tokens, significantly more than the Chinchilla-optimal amount for their sizes. For instance, the Chinchilla-optimal dataset for the 8B model is 200 billion tokens, yet performance continued to scale log-linearly up to 75 times that amount. This empirical finding highlighted the benefits of training on larger datasets than previously considered optimal. The 70B model was still learning at the end of training, but Meta decided to stop to allocate GPU resources elsewhere. The models were trained using Adam optimizer and other techniques common in deep learning.
Release and Availability
LLaMA 3 was released on April 18, 2024, with weights available for both foundation and instruction-tuned versions. Unlike the original LLaMA, which required case-by-case approval, LLaMA 3 was made available under a license permitting commercial use, though with acceptable use policies that some argue prevent it from being truly open source. The release included a dedicated website for Meta AI, an assistant built on LLaMA 3, and integration into Facebook and WhatsApp. Meta also announced future plans for larger models and expanded capabilities.
Performance and Reception
In Meta's internal testing, LLaMA 3 70B outperformed Google DeepMind's Gemini Pro 1.5 and Anthropic's Claude 3 Sonnet on most benchmarks. The 8B model was reported by Mark Zuckerberg to be nearly as powerful as the largest Llama 2 model. The release was well received in the AI community, with many praising the open availability of high-performing models. However, some commentators noted that the license restrictions meant it was not fully open source, a point also raised by the Open Source Initiative. The success of LLaMA 3 contributed to the proliferation of open-weight models and influenced subsequent releases, including Llama 3.1 in July 2024 and Llama 4 in April 2025.
Impact and Legacy
LLaMA 3's release reinforced the trend of open-weight models challenging proprietary systems. Its training data scale and performance demonstrated the importance of data quantity in model development. The model's availability on platforms like Hugging Face and through cloud services such as AWS and Azure facilitated widespread adoption. In April 2026, Meta Superintelligence Labs released Muse Spark as a replacement for Llama, marking the end of the Llama series. Nevertheless, LLaMA 3 remains a significant milestone in the evolution of artificial intelligence and machine learning.