Llion Jones is a Welsh machine learning researcher and entrepreneur. He is best known as a co-author of the 2017 paper "Attention Is All You Need," which introduced the Transformer (architecture) architecture, a foundational model for modern large language models. In 2023, he co-founded Sakana AI, a Tokyo-based startup focused on nature-inspired AI, where he serves as Chief Technology Officer.
Jones was born in 1984 in Bangor, Wales, and raised in Bangor and Abergynolwyn in Gwynedd. He attended Coleg Meirion-Dwyfor before pursuing higher education at the University of Birmingham, where he earned a bachelor's degree in artificial intelligence and computer science, followed by a master's degree in advanced computer science. He graduated in 2009.
Early Career and Google
After graduating, Jones joined Google's YouTube division as a software engineer in 2011 or 2012. In 2015, he transitioned to Google Research, where he worked on machine intelligence and natural language processing. His work at Google Research placed him at the center of a team that would produce one of the most influential papers in the field of artificial intelligence.
The Transformer Paper
In 2017, Jones co-authored "Attention Is All You Need" alongside Jakob Uszkoreit, Lukasz Kaiser, Niki Parmar, Barret Zoph, and others. The paper introduced the transformer architecture, which relies entirely on self-attention mechanisms rather than recurrent or convolutional layers. This design allowed for more efficient parallel processing and significantly improved performance on sequence-to-sequence tasks. The transformer became the backbone of subsequent AI systems, including BERT, GPT, and other large language models, and has been widely adopted in machine learning and deep learning research.
Criticism of AI Research Focus
Jones has been vocal about the narrow scope of AI research, particularly criticizing Google's heavy focus on large language models at the expense of other approaches. He has argued that the field's concentration on scaling up language models has limited exploration of alternative architectures and learning paradigms. This perspective influenced his later work with Sakana AI, which seeks to draw inspiration from natural systems and collective intelligence.
Founding Sakana AI
In 2023, Jones left Google Japan to co-found Sakana AI with David Ha, a former Google researcher and Stability AI officer. They were joined by Ren Ito. The company's name, "Sakana," means "fish" in Japanese, reflecting its mission to develop AI that mimics the adaptive and collective behaviors of natural organisms. Sakana AI aims to create more efficient and flexible AI systems by leveraging principles such as swarm intelligence and evolutionary algorithms. Jones took on the role of Chief Technology Officer, overseeing the technical direction and research strategy.
Impact and Recognition
Jones's contributions to the transformer paper have had a lasting impact on the field of artificial intelligence. The architecture he helped create underpins many of the most prominent AI products and services, including OpenAI's GPT series, Google DeepMind's language models, and various generative AI applications. His work has been cited extensively in academic literature and has influenced the development of neural networks beyond natural language processing, including in computer vision and audio processing.
Personal Life and Interests
Jones is known for his interest in the intersection of AI and biology, often drawing parallels between natural systems and computational models. He has spoken about the importance of interdisciplinary research and the potential for AI to be inspired by the efficiency and adaptability of living organisms. His Welsh heritage and upbringing in rural Gwynedd have been noted in interviews as shaping his perspective on technology and community.
Later Developments
As of 2025, Sakana AI has continued to grow, attracting attention from investors and researchers interested in alternative AI paradigms. Jones remains active in the AI community, contributing to discussions on the future of the field and advocating for a broader research agenda beyond large language models. His work with Sakana AI represents a significant departure from mainstream approaches, focusing on smaller, more adaptable models that can operate efficiently in resource-constrained environments.
Legacy
The transformer architecture introduced by Jones and his colleagues has been described as a turning point in AI research, enabling breakthroughs in machine learning and deep learning. Its influence extends to major tech companies and research institutions, including Google DeepMind, OpenAI, Anthropic, and academic labs such as Stanford AI Lab and MIT CSAIL. Jones's role in this development, along with his subsequent entrepreneurial efforts, positions him as a notable figure in the ongoing evolution of artificial intelligence.
References
- "Attention Is All You Need" (2017) - Vaswani et al.
- Sakana AI official website and press releases
- Interviews and public talks by Llion Jones