The TPU v2 is the second-generation Tensor Processing Unit, a custom application-specific integrated circuit (ASIC) developed by Google for accelerating neural network machine learning workloads. Announced in May 2017, it succeeded the original TPU and introduced significant architectural improvements, most notably the ability to perform floating-point calculations and the inclusion of the bfloat16 format, which was invented by Google Brain. This made the TPU v2 suitable for both training and inference of machine learning models, unlike its predecessor which was limited to integer operations. The TPU v2 was made available to third parties through Google Cloud's Cloud TPU service, and it has been used extensively in Google's own production systems, including for transformer-based models and large language models.
The TPU v2 was designed to address the memory bandwidth limitations of the first-generation TPU. By incorporating 16 GB of High Bandwidth Memory (HBM), the second-generation design increased memory bandwidth to 600 GB/s and delivered 45 teraFLOPS of performance per chip. These chips were then arranged into four-chip modules, achieving a combined performance of 180 teraFLOPS. Furthermore, 64 of these modules were assembled into 256-chip pods, providing 11.5 petaFLOPS of total performance. This scalable architecture allowed Google to deploy TPU v2 systems for large-scale machine learning tasks, including those involving neural networks and deep learning.
Development and History
The TPU v2's development was led by Amir Salek, who founded and headed Google's Custom Silicon group, and Norman P. Jouppi served as the tech lead and principal architect for the overall TPU program. The original TPU was deployed in production in just 15 months, and the v2 built upon that foundation. According to Jonathan Ross, one of the original TPU engineers and later founder of Groq, the TPU design, which uses a systolic array architecture, was selected from among three competing AI accelerator proposals at Google. The TPU v2 was the industry's first production deep-learning training chip, marking a milestone in custom silicon for AI. Broadcom co-developed the TPU v2, translating Google's architecture into manufacturable silicon and managing fabrication through foundries like TSMC.
Technical Specifications
The TPU v2 is a matrix multiplication engine that supports both integer and floating-point arithmetic. Its key innovation was the introduction of the bfloat16 format, a 16-bit floating-point representation that provides a wider dynamic range than traditional 16-bit formats while requiring less memory and bandwidth. This made it particularly effective for training deep neural networks. Each TPU v2 chip contains a systolic array of multipliers and accumulators, along with 16 GB of HBM providing 600 GB/s of bandwidth. The chips are interconnected in four-chip modules, and these modules are combined into larger pods. The 256-chip pod configuration delivers 11.5 petaFLOPS, enabling massive parallel computation for tasks like training transformers and other large language models.
Applications and Use
Google used TPU v2 systems in a variety of production applications. They were employed in the AlphaGo versus Lee Sedol series of Go matches, as well as in the AlphaZero system, which learned to play chess, shogi, and Go from game rules alone. TPUs were also used for Google Street View text processing, enabling the extraction of all text in the Street View database in less than five days. In Google Photos, a single TPU could process over 100 million photos per day. Additionally, TPU v2s were used in RankBrain, Google's system for providing search results. Through the Cloud TPU service on Google Cloud, third parties could access TPU v2s for their own machine learning workloads, supporting frameworks like TensorFlow, JAX, and PyTorch.
Comparison with GPUs
Compared to GPUs, TPUs are designed for high-volume, low-precision computation with more input/output operations per joule. They lack hardware for rasterization or texture mapping, focusing instead on matrix operations. TPUs are well suited for convolutional neural networks (CNNs), while GPUs have benefits for some fully connected neural networks, and CPUs can have advantages for recurrent neural networks (RNNs). In workflows involving transformer-based neural networks for large language models, TPUs are often used for inference, while GPUs are frequently used for training. This division of labor reflects the different strengths of each processor type, with TPUs excelling in the specific patterns of computation found in many modern AI models.
Legacy and Impact
The TPU v2 established a template for subsequent generations of Google's TPU hardware, including the TPU v3, which was announced on May 8, 2018, with twice the performance per chip and pods with four times as many chips. The introduction of bfloat16 had a lasting impact on the AI hardware ecosystem, influencing other accelerator designs. As of 2025, Google Cloud generates significant product revenues from the sale of TPU systems, and Google has been in talks with various "neoclouds" and companies like Meta about deploying TPUs in their data centers. The TPU v2 remains a foundational platform in the history of artificial intelligence hardware, demonstrating the value of custom silicon for machine learning.