"Attention Is All You Need" is the 2017 research paper that introduced the Transformer architecture, a neural network design built entirely around attention mechanisms rather than recurrence or convolution. Published as an arXiv preprint on June 12, 2017 and presented at the NeurIPS conference later that year, it was written by eight researchers at Google: Ashish Vaswani, Noam Shazeer, Aidan Gomez, Illia Polosukhin, Jakob Uszkoreit, Llion Jones, Lukasz Kaiser, and Niki Parmar. It is among the most cited papers in computer science and is widely regarded as the single most consequential publication in the development of modern large language models.
Background
Before the Transformer, state-of-the-art Natural language processing systems for tasks such as machine translation relied on recurrent architectures, often LSTMs, processing text sequentially and augmented with attention layers borrowed from earlier sequence-to-sequence work. Sequential processing was slow to train and struggled to capture long-range dependencies. The authors proposed replacing recurrence entirely with self-attention, letting the model weigh the relevance of every token in a sequence to every other token in parallel, which sped up training on GPUs and improved the modeling of long-range structure.
Architecture and impact
The paper's design, an encoder-decoder stack of self-attention and feed-forward layers with positional encodings substituting for recurrence, became the template for nearly every major model that followed. BERT adapted the encoder side for language understanding, while GPT-style models adopted decoder-only variants trained through next-token prediction. The architecture proved unusually easy to scale, feeding directly into the Scaling laws research that showed predictable performance gains from more data, parameters, and compute, and it later underpinned foundation models across text, vision, audio, and multimodal domains.
Legacy
A frequently repeated fact about the paper is that all eight original authors eventually left Google to found or join other AI companies. Noam Shazeer co-founded Character.AI before returning to Google, and Aidan Gomez became chief executive of Cohere. The paper's citation count grew into the tens of thousands within less than a decade of publication, and the Transformer remains, as of the mid-2020s, the dominant architecture behind frontier models, even as researchers continue exploring alternatives such as state space models for specific efficiency tradeoffs.