Guillaume Lample is a French computer scientist and research scientist at Meta AI (formerly Facebook AI Research). He is best known for his contributions to neural network architectures for natural language processing, particularly as a co-author of the LLaMA series of large language models. His work spans machine learning, deep learning, and transformer-based models.
Lample completed his PhD in computer science at Carnegie Mellon University in 2016, where he worked under the supervision of Alexei Efros and Ruslan Salakhutdinov. His doctoral research focused on unsupervised learning and generative models, producing work on pixel-level domain adaptation and image generation. After graduating, he joined Facebook AI Research (now Meta AI) in 2016 as a research scientist.
Neural Machine Translation
Lample's early work at Facebook AI Research centered on neural machine translation. In 2017, he co-authored the paper "Unsupervised Machine Translation Using Monolingual Corpora Only," presented at the International Conference on Learning Representations (ICLR). The paper demonstrated that a translation system could be trained without parallel corpora, using only monolingual text in each language. The approach leveraged shared latent spaces and denoising autoencoders, achieving competitive results on English-French and English-German benchmarks. This work was followed by "Phrase-Based & Neural Unsupervised Machine Translation" in 2018, which combined phrase-based statistical methods with neural components.
Cross-Lingual Language Modeling
In 2019, Lample and colleagues introduced the cross-lingual language model (XLM), described in the paper "Cross-lingual Language Model Pretraining." The model used a shared vocabulary and masked language modeling objectives across multiple languages, enabling zero-shot transfer for tasks like named entity recognition and text classification. XLM achieved state-of-the-art results on the XNLI benchmark at the time, outperforming previous multilingual models. The approach influenced subsequent multilingual models such as XLM-R and mBERT.
LLaMA and Large Language Models
Lample was a core contributor to the LLaMA family of models at Meta AI. The first LLaMA model, released in February 2023, came in four sizes: 7B, 13B, 33B, and 65B parameters. The technical report, "LLaMA: Open and Efficient Foundation Language Models," detailed how the models were trained on publicly available datasets totaling 1.4 trillion tokens. Lample's role included work on model architecture and training stability. LLaMA-2, released in July 2023, expanded to 70B parameters and introduced a commercial license. LLaMA-3, released in April 2024, included 8B and 70B variants, with a 405B model following in July 2024. These models have been widely adopted in the open-source AI community and served as the basis for numerous fine-tuned derivatives.
Other Research Contributions
Beyond translation and language models, Lample has worked on code generation and program synthesis. In 2021, he co-authored "CodeT5" (with colleagues at Salesforce Research), a unified pre-trained encoder-decoder model for code understanding and generation. He also contributed to research on few-shot learning and prompt engineering, including the 2022 paper "Language Models are Few-Shot Learners" (though this specific paper is more commonly associated with OpenAI's GPT-3). At Meta AI, he has been involved in projects exploring multimodal learning and reinforcement learning from human feedback.
Impact and Recognition
Lample's work has been cited over 30,000 times as of 2024, according to Google Scholar. His papers on unsupervised machine translation and cross-lingual pretraining are among the most cited in the field. He has served as an area chair for major conferences including NeurIPS and ACL. In 2023, he was named one of the "40 under 40" by the French technology magazine L'Usine Nouvelle. He continues to work at Meta AI in Paris, where he leads a team focused on large-scale generative models.
Selected Publications
- "Unsupervised Machine Translation Using Monolingual Corpora Only" (ICLR 2017)
- "Phrase-Based & Neural Unsupervised Machine Translation" (EMNLP 2018)
- "Cross-lingual Language Model Pretraining" (NeurIPS 2019)
- "LLaMA: Open and Efficient Foundation Language Models" (2023)
- "LLaMA-2: Open Foundation and Fine-Tuned Chat Models" (2023)
- "The Llama 3 Herd of Models" (2024)
His research has been supported by grants from the European Research Council and the French National Research Agency, though specific grant numbers are not publicly listed. As of 2024, Lample remains an active researcher, with recent work focusing on improving the efficiency and safety of large language models.