Deep learning is a branch of Machine learning that uses artificial Neural networks with many stacked layers to learn hierarchical representations directly from raw data, such as pixels, audio waveforms, or text, without requiring hand-engineered features. The word "deep" refers to the number of layers between input and output.
History
The mathematical foundations of deep learning date to the 1980s, when Backpropagation was popularized as a way to train multi-layer networks. Progress stalled through the 1990s and 2000s due to limited data, limited compute, and competition from other statistical methods, a period sometimes described as part of a broader AI winter for connectionist approaches specifically. The field's modern era is usually dated to 2012, when AlexNet, a deep Convolutional neural network trained on graphics processing units, won the ImageNet competition by a wide margin over methods based on hand-crafted features. Researchers Geoffrey Hinton, Yann LeCun, and Yoshua Bengio, later called the field's founding trio, received the 2018 Turing Award for laying its groundwork; Hinton and physicist John Hopfield shared the 2024 Nobel Prize in Physics for related foundational work.
Architecture
Early deep learning success came from convolutional networks for images and Recurrent neural networks, including the LSTM variant, for sequential data such as text and speech. The 2017 introduction of the Transformer (architecture) architecture, which processes sequences in parallel using Attention mechanisms rather than step-by-step recurrence, largely displaced recurrent networks for language tasks and became the basis of modern Large language models. Networks are trained by minimizing a loss function through variants of Gradient descent, with backpropagation used to compute how each parameter should change.
Why it works at scale
Deep networks benefit disproportionately from more data and more compute compared with earlier statistical methods, a relationship later formalized as scaling laws. This scaling behavior, combined with the parallel processing capability of GPU (in AI) hardware originally built for graphics rendering, allowed practitioners to train ever-larger models through the 2010s and 2020s. Nvidia's dominance in AI-capable GPUs and its CUDA software platform became a central piece of industry infrastructure as a result.
Impact and criticism
Deep learning drove rapid progress in computer vision, speech recognition, machine translation, and, most visibly to the public, text and image generation. Critics note that deep networks remain largely opaque, function as statistical pattern matchers whose failures can be difficult to predict, and require enormous energy and data resources, raising both environmental and copyright concerns. Proponents counter that the same scaling recipe has repeatedly produced capabilities that were not explicitly designed for, a pattern central to ongoing debates about the field's trajectory toward more general intelligence.