# Deep learning

Deep learning is a subfield of machine learning that trains neural networks with many layers to learn hierarchical representations directly from raw data, powering modern computer vision, speech, and language systems.

Deep learning is a branch of [machine-learning](https://www.wikiprompt.org/wiki/machine-learning) that uses artificial [neural-network](https://www.wikiprompt.org/wiki/neural-network)s with many stacked layers to learn hierarchical representations directly from raw data, such as pixels, audio waveforms, or text, without requiring hand-engineered features. The word "deep" refers to the number of layers between input and output.

## History

The mathematical foundations of deep learning date to the 1980s, when [backpropagation](https://www.wikiprompt.org/wiki/backpropagation) was popularized as a way to train multi-layer networks. Progress stalled through the 1990s and 2000s due to limited data, limited compute, and competition from other statistical methods, a period sometimes described as part of a broader AI winter for connectionist approaches specifically. The field's modern era is usually dated to 2012, when [alexnet](https://www.wikiprompt.org/wiki/alexnet), a deep [convolutional-neural-network](https://www.wikiprompt.org/wiki/convolutional-neural-network) trained on graphics processing units, won the ImageNet competition by a wide margin over methods based on hand-crafted features. Researchers [geoffrey-hinton](https://www.wikiprompt.org/wiki/geoffrey-hinton), [yann-lecun](https://www.wikiprompt.org/wiki/yann-lecun), and [yoshua-bengio](https://www.wikiprompt.org/wiki/yoshua-bengio), later called the field's founding trio, received the 2018 Turing Award for laying its groundwork; Hinton and physicist John Hopfield shared the 2024 Nobel Prize in Physics for related foundational work.

## Architecture

Early deep learning success came from convolutional networks for images and [recurrent-neural-network](https://www.wikiprompt.org/wiki/recurrent-neural-network)s, including the LSTM variant, for sequential data such as text and speech. The 2017 introduction of the [transformer](https://www.wikiprompt.org/wiki/transformer) architecture, which processes sequences in parallel using [attention-mechanism](https://www.wikiprompt.org/wiki/attention-mechanism)s rather than step-by-step recurrence, largely displaced recurrent networks for language tasks and became the basis of modern [large-language-model](https://www.wikiprompt.org/wiki/large-language-model)s. Networks are trained by minimizing a loss function through variants of [gradient-descent](https://www.wikiprompt.org/wiki/gradient-descent), with backpropagation used to compute how each parameter should change.

## Why it works at scale

Deep networks benefit disproportionately from more data and more compute compared with earlier statistical methods, a relationship later formalized as scaling laws. This scaling behavior, combined with the parallel processing capability of [gpu](https://www.wikiprompt.org/wiki/gpu) hardware originally built for graphics rendering, allowed practitioners to train ever-larger models through the 2010s and 2020s. Nvidia's dominance in AI-capable GPUs and its CUDA software platform became a central piece of industry infrastructure as a result.

## Impact and criticism

Deep learning drove rapid progress in computer vision, speech recognition, machine translation, and, most visibly to the public, text and image generation. Critics note that deep networks remain largely opaque, function as statistical pattern matchers whose failures can be difficult to predict, and require enormous energy and data resources, raising both environmental and copyright concerns. Proponents counter that the same scaling recipe has repeatedly produced capabilities that were not explicitly designed for, a pattern central to ongoing debates about the field's trajectory toward more general intelligence.

---
Source: https://www.wikiprompt.org/wiki/deep-learning
License: CC BY-SA 4.0 (https://creativecommons.org/licenses/by-sa/4.0/)
Last updated: 2026-09-02T20:28:38.664477+00:00