A graphics processing unit (GPU) is a specialized processor originally designed to render images and video by performing many simple calculations in parallel. That same architecture, thousands of small cores executing the same operation on different pieces of data simultaneously, turned out to be extremely well suited to the matrix and vector math underlying neural networks, and GPUs became the dominant hardware for training and running deep learning models.
Why GPUs suit AI
Training a neural network involves repeated multiplication and addition of large matrices, operations that can be broken into many independent pieces and executed simultaneously. A central processing unit has a small number of powerful cores optimized for sequential, general-purpose tasks; a GPU has thousands of simpler cores optimized for exactly this kind of parallel, repetitive arithmetic. This mismatch made GPUs, originally built for the parallel task of computing pixel colors across a screen, far faster than CPUs at the gradient descent computations used in training and at the matrix multiplications used in inference.
History
Nvidia, founded in 1993, established the modern GPU category with its consumer graphics cards, then opened GPUs to general-purpose computing in 2006 with CUDA, a programming platform that let developers write non-graphics code for the GPU's parallel cores. The turning point for AI came in 2012, when AlexNet, trained on two Nvidia GTX 580 GPUs, won the ImageNet competition by a wide margin, demonstrating that GPU-accelerated deep learning could dramatically outperform prior computer vision methods and triggering a shift across the field toward GPU training. Nvidia subsequently built a line of data-center GPUs specifically for AI, including the V100 (2017), A100 (2020), H100 (2022), and Blackwell-generation chips (2024), each substantially faster than the last at the low-precision matrix math used in deep learning.
Market and impact
By the AI boom of the early-to-mid 2020s, demand for Nvidia's data-center GPUs, driven by the compute needs of training large language models and other foundation models, made Nvidia briefly the world's most valuable public company, with chips such as the H100 in persistent short supply. This concentrated Nvidia's position as the dominant supplier of AI training hardware, reinforced by CUDA's status as the de facto software standard that most AI frameworks, including PyTorch and TensorFlow, are optimized for. Competitors, including AMD, Intel, and custom accelerators such as Google's TPU, have challenged Nvidia's dominance, and major AI labs have increasingly designed their own chips, but as of 2025 GPUs, and Nvidia's in particular, remained the primary engine of large-scale AI training and a frequent bottleneck and cost driver for the industry, prompting export controls by the United States government restricting sales of its most advanced chips to China.