Thomas Scialom is a research scientist at Meta AI (now part of AI research at Meta), recognized for his work on large language models and machine learning. He is best known as a co-author of Llama 2, a family of open-source large language models released in 2023, and for his research on improving the factual accuracy and reasoning capabilities of these models through techniques such as retrieval-augmented generation and [[reinforcement learning from human feedback] (RLHF). His work has influenced both academic research and practical applications in generative AI.
Scialom's research sits at the intersection of deep learning, natural language processing, and neural network architectures. He has contributed to multiple projects at Meta AI, including the development of Llama and its successors, and has published extensively on topics like text summarization, question answering, and model alignment. His contributions are part of a broader effort within the AI community to create more reliable and controllable language models.
Early Career and Education
Thomas Scialom completed his doctoral studies in computer science, focusing on natural language processing and deep learning. His PhD research, conducted in collaboration with academic institutions and industry labs, explored novel approaches to text summarization and Sequence-to-Sequence (Seq2Seq) models. He later joined Meta AI (formerly Facebook AI Research) as a research scientist, where he began working on large-scale language models.
During his early career, Scialom contributed to several foundational papers on transformer-based architectures, particularly in the context of summarization and dialogue systems. His work often emphasized the importance of evaluation metrics and human evaluation in assessing model performance, a theme that continued in his later research on RLHF.
Contributions to Llama 2
Scialom was a key contributor to Llama 2, released by Meta AI in July 2023. Llama 2 is a collection of pretrained and fine-tuned large language models ranging from 7 billion to 70 billion parameters. Unlike its predecessor, Llama 2 was made available for commercial use, marking a significant shift in the open-source AI landscape. Scialom co-authored the accompanying technical report, which detailed the model's architecture, training data, and alignment process.
A notable aspect of Llama 2 was its use of reinforcement learning from human feedback to improve helpfulness and safety. Scialom's research on RLHF, including the development of reward models and proximal policy optimization techniques, was instrumental in this effort. The model's release spurred widespread adoption and influenced subsequent open-source models from other organizations.
Research on Retrieval-Augmented Generation
Beyond Llama 2, Scialom has made significant contributions to retrieval-augmented generation (RAG), a method that combines information retrieval with generative models to improve factual accuracy. His work in this area, including the paper "Recitation-Augmented Language Models" (2022), demonstrated how prompting models to recite relevant passages before generating answers can enhance performance on knowledge-intensive tasks.
This line of research addressed a key limitation of large language models: their tendency to produce plausible but incorrect information, often called Hallucination (AI). By integrating external knowledge sources, Scialom's methods aimed to ground model outputs in verifiable facts, a direction that has become central to many AI applications, including question answering and fact-checking.
Impact and Recognition
Scialom's work has been widely cited in the AI community, and he is a frequent speaker at conferences such as NeurIPS, ICML, and ACL. His contributions to open-source AI, particularly through Llama 2, have been praised for democratizing access to advanced language models. He has also collaborated with researchers from other leading institutions, including Google DeepMind and OpenAI, on topics related to model evaluation and alignment.
As of 2024, Scialom continues to work at Meta AI, focusing on improving the reasoning and safety of large language models. His research remains influential in shaping the direction of artificial intelligence, particularly in the development of models that are both powerful and reliable.
See Also
- Large language model
- Reinforcement learning from human feedback
- Retrieval-augmented generation
- Meta AI