# Meta LLaMA 3 Release

Meta released Llama 3, a family of open-weight large language models, on April 18, 2024, in 8B and 70B parameter sizes, trained on 15 trillion tokens. It outperformed several commercial rivals on benchmarks and marked a shift toward scaling laws beyond Chinchilla-optimal data.

Meta released Llama 3, a family of large language models (LLMs), on April 18, 2024, as part of its ongoing Llama series. The release included two model sizes: 8 billion and 70 billion parameters, both pre-trained on approximately 15 trillion tokens of text from publicly available sources. Instruction-tuned variants were fine-tuned on public instruction datasets and over 10 million human-annotated examples. Meta positioned Llama 3 as a significant step in open-weight AI, with the 70B model outperforming several commercial rivals on standard benchmarks at the time of release.

The Llama series began in February 2023 with Llama 1, initially released only to researchers under a non-commercial license. After an unauthorized leak via BitTorrent in March 2023, Meta shifted strategy with Llama 2 in July 2023, offering weights for broader commercial use under a custom license. Llama 3 continued this trajectory, emphasizing accessibility and performance. The release coincided with Meta AI, an assistant built on Llama, available via a dedicated website and integrated into Facebook and WhatsApp.

## Model Architecture and Training

Llama 3 models use a standard transformer architecture with improvements over prior versions. The 8B and 70B models were trained on 15 trillion tokens, a dataset 75 times larger than the Chinchilla-optimal amount for the 8B size (200 billion tokens). Despite exceeding the optimal data point, performance continued to scale log-linearly, challenging conventional scaling laws. Meta reported that the 70B model was still improving at the end of training, but the team chose to stop to allocate GPU resources elsewhere.

The training data was curated from publicly available sources, with an emphasis on quality and diversity. Instruction fine-tuning used a combination of public datasets and human-annotated examples, enhancing the models' ability to follow complex instructions. Meta also announced plans to make future versions multilingual, multimodal, and better at coding and reasoning, with an increased context window.

## Performance and Benchmarks

In April 2024, Meta's internal testing showed that Llama 3 70B outperformed Gemini Pro 1.5 and Claude 3 Sonnet on most benchmarks, including reasoning, coding, and general knowledge tasks. The 8B model was reported by Mark Zuckerberg to be nearly as powerful as the largest Llama 2 model (70B), demonstrating significant efficiency gains. These results positioned Llama 3 as a competitive alternative to proprietary models from companies like [openai](https://www.wikiprompt.org/wiki/openai), [anthropic](https://www.wikiprompt.org/wiki/anthropic), and [google-deepmind](https://www.wikiprompt.org/wiki/google-deepmind).

The models were evaluated on standard NLP benchmarks, with the 70B variant achieving state-of-the-art results among open-weight models at the time. The scaling behavior observed - continued improvement beyond Chinchilla-optimal data - suggested that larger datasets could yield further gains, influencing subsequent research in [machine-learning](https://www.wikiprompt.org/wiki/machine-learning) and [deep-learning](https://www.wikiprompt.org/wiki/deep-learning).

## Release and Ecosystem

Llama 3 was released under a custom license permitting commercial use, though with an acceptable use policy that restricted certain applications, leading to debates about whether it qualified as open source. The weights were made available for download, and the models were integrated into various platforms, including [amazon-web-services](https://www.wikiprompt.org/wiki/amazon-web-services), [azure](https://www.wikiprompt.org/wiki/azure), [google-cloud](https://www.wikiprompt.org/wiki/google-cloud), and [oracle-cloud](https://www.wikiprompt.org/wiki/oracle-cloud). Specialized hardware providers like [groq](https://www.wikiprompt.org/wiki/groq) and [samba-nova](https://www.wikiprompt.org/wiki/samba-nova) optimized inference for Llama 3, enabling faster deployment.

The release spurred a wave of fine-tuned variants and tools, similar to the ecosystem that emerged around Llama 2. Developers used techniques like [rlaif](https://www.wikiprompt.org/wiki/rlaif) and [curriculum-learning](https://www.wikiprompt.org/wiki/curriculum-learning) to adapt the models for specific domains. The open-weight approach contrasted with closed models from competitors, fostering innovation in [generative-ai](https://www.wikiprompt.org/wiki/generative-ai) applications.

## Subsequent Developments

Llama 3.1 was released on July 23, 2024, introducing a 405B parameter model and expanding the context window. This version further solidified Llama's position in the AI landscape. The series continued with Llama 4 in April 2025, and in April 2026, Meta Superintelligence Labs released Muse Spark as a replacement. The evolution of Llama influenced the broader field, with researchers studying scaling laws and training efficiency, as seen in work on [transformer](https://www.wikiprompt.org/wiki/transformer) architectures and [loss-functions](https://www.wikiprompt.org/wiki/loss-functions).

The release also highlighted the role of large-scale compute, with Meta leveraging partnerships with chipmakers like [amd](https://www.wikiprompt.org/wiki/amd), [intel](https://www.wikiprompt.org/wiki/intel), and [tsmc](https://www.wikiprompt.org/wiki/tsmc) to train models efficiently. The success of Llama 3 contributed to discussions about the future of [artificial-intelligence](https://www.wikiprompt.org/wiki/artificial-intelligence) and the balance between open and proprietary development.

---
Source: https://www.wikiprompt.org/wiki/meta-llama-3-release
License: CC BY-SA 4.0 (https://creativecommons.org/licenses/by-sa/4.0/)
Last updated: 2026-09-09T02:02:05.054841+00:00