The Tensor Processing Unit (TPU) is an application-specific integrated circuit designed by Google specifically to accelerate machine learning workloads, particularly the matrix multiplications at the core of neural network training and inference. Unlike a general-purpose GPU, a TPU is purpose-built for tensor operations and is not designed to render graphics.
History
Google began deploying TPUs internally in 2015 to accelerate services such as Search ranking, Street View text recognition, and AlphaGo's move evaluation during its matches, and announced the chip publicly at its 2016 I/O conference. The first-generation TPU focused on inference for already-trained models; the second generation, released in 2017, added support for training, and Google made TPUs available to external customers through Google Cloud. Successive generations, TPU v3 (2018), v4 (2021), v5 (2023), and Trillium (2024), each increased performance and memory, and by the mid-2020s TPUs formed large interconnected pods of thousands of chips used to train Google's own foundation models, including its Gemini family, as well as being rented out to external customers via Google Cloud.
Design and comparison to GPUs
TPUs are built around a systolic array architecture, a grid of processing elements that pass data to neighboring elements in a fixed rhythm, which is highly efficient for the repeated matrix multiplications used in deep learning but far less flexible than a GPU for general-purpose parallel computing. This specialization gives TPUs an efficiency advantage per watt and per dollar for large-scale training and serving of neural networks, at the cost of being less adaptable to non-tensor workloads. Where Nvidia's GPUs run on the widely adopted CUDA software platform, TPUs are programmed primarily through TensorFlow, and later PyTorch and JAX, Google's own numerical computing library, giving them a smaller but growing software ecosystem.
Significance
TPUs represent one of the clearest examples of a major AI lab vertically integrating its own chip design rather than relying solely on merchant silicon from suppliers like Nvidia, a strategy later echoed by other companies designing custom AI accelerators, such as Amazon's Trainium and Meta's MTIA, as demand for large language model training and inference capacity grew during the AI boom of the 2020s. Because TPUs are primarily available through Google Cloud rather than sold as standalone hardware, they have remained a smaller share of the overall AI accelerator market than GPUs, but they underpin Google's internal AI infrastructure, including training runs for its largest models, and are cited by Google as a key reason it can compete with rivals that depend more heavily on external GPU supply.