# Deep learning

Deep learning is a subfield of machine learning that uses multilayered neural networks to learn hierarchical representations directly from raw data, driving advances in computer vision, speech, and language processing.

Deep learning is a branch of [machine-learning](https://www.wikiprompt.org/wiki/machine-learning) that uses artificial [neural-network](https://www.wikiprompt.org/wiki/neural-network)s with many stacked layers to learn hierarchical representations directly from raw data, such as pixels, audio waveforms, or text, without requiring hand-engineered features. The word "deep" refers to the number of layers between input and output, with systems typically having a credit assignment path (CAP) depth greater than two. Deep learning can be supervised, semi-supervised, or unsupervised, and draws inspiration from but does not aim to model biological brain function.

## History

The mathematical foundations of deep learning date to the 1980s, when [backpropagation](https://www.wikiprompt.org/wiki/backpropagation) was popularized as a way to train multi-layer networks. Progress stalled through the 1990s and 2000s due to limited data, limited compute, and competition from other statistical methods, a period sometimes described as part of a broader AI winter for connectionist approaches specifically. The field's modern era is usually dated to 2012, when [alexnet](https://www.wikiprompt.org/wiki/alexnet), a deep [convolutional-neural-network](https://www.wikiprompt.org/wiki/convolutional-neural-network) trained on graphics processing units, won the ImageNet competition by a wide margin over methods based on hand-crafted features. Researchers [geoffrey-hinton](https://www.wikiprompt.org/wiki/geoffrey-hinton), [yann-lecun](https://www.wikiprompt.org/wiki/yann-lecun), and [yoshua-bengio](https://www.wikiprompt.org/wiki/yoshua-bengio), later called the field's founding trio, received the 2018 Turing Award for laying its groundwork; Hinton and physicist John Hopfield shared the 2024 Nobel Prize in Physics for related foundational work.

The term "deep learning" was introduced to the machine learning community by Rina Dechter in 1986, and to artificial neural networks by Igor Aizenberg and colleagues in 2000 in the context of Boolean threshold neurons.

## Architecture

Early deep learning success came from convolutional networks for images and [recurrent-neural-network](https://www.wikiprompt.org/wiki/recurrent-neural-network)s, including the LSTM variant, for sequential data such as text and speech. Common architectures also include fully connected networks, deep belief networks, generative adversarial networks ([gans](https://www.wikiprompt.org/wiki/gans)), transformers, and [neural-radiance-fields](https://www.wikiprompt.org/wiki/neural-radiance-fields). The 2017 introduction of the [transformer](https://www.wikiprompt.org/wiki/transformer) architecture, which processes sequences in parallel using [attention-mechanism](https://www.wikiprompt.org/wiki/attention-mechanism)s rather than step-by-step recurrence, largely displaced recurrent networks for language tasks and became the basis of modern [large-language-model](https://www.wikiprompt.org/wiki/large-language-model)s. Networks are trained by minimizing a loss function through variants of [gradient-descent](https://www.wikiprompt.org/wiki/gradient-descent), with backpropagation used to compute how each parameter should change. Deep learning models can be constructed with a greedy layer-by-layer method, and can be applied to unsupervised learning tasks.

## Why it works at scale

Deep networks benefit disproportionately from more data and more compute compared with earlier statistical methods, a relationship later formalized as scaling laws. This scaling behavior, combined with the parallel processing capability of [gpu](https://www.wikiprompt.org/wiki/gpu) hardware originally built for graphics rendering, allowed practitioners to train ever-larger models through the 2010s and 2020s. Nvidia's dominance in AI-capable GPUs and its CUDA software platform became a central piece of industry infrastructure as a result.

## Interpretations

Deep neural networks are generally interpreted in terms of the universal approximation theorem or probabilistic inference. The classic universal approximation theorem states that feedforward networks with a single hidden layer of finite size can approximate continuous functions; the first proof was published by George Cybenko in 1989 for sigmoid activations and generalized to multi-layer architectures by Kurt Hornik in 1991. Recent work extended universal approximation to unbounded activations such as the rectified linear unit; for ReLU networks, Lu et al. proved that width strictly larger than the input dimension suffices for universal approximation, while smaller or equal width does not. The probabilistic interpretation, which considers activation nonlinearities as cumulative distribution functions, led to the introduction of dropout as a regularizer. This interpretation was introduced by researchers including Hopfield, Widrow, and Narendra, and popularized in surveys such as Bishop's.

## Impact and criticism

Deep learning drove rapid progress in [computer-vision](https://www.wikiprompt.org/wiki/computer-vision), [speech-recognition](https://www.wikiprompt.org/wiki/speech-recognition), [machine-translation](https://www.wikiprompt.org/wiki/machine-translation), [natural-language-processing](https://www.wikiprompt.org/wiki/natural-language-processing), and, most visibly to the public, text and image generation. The field has also been applied to bioinformatics, drug design, medical image analysis, climate science, material inspection, and board game programs, where results have matched or surpassed human expert performance in some cases. Critics note that deep networks remain largely opaque, function as statistical pattern matchers whose failures can be difficult to predict, and require enormous energy and data resources, raising both environmental and copyright concerns. Proponents counter that the same scaling recipe has repeatedly produced capabilities that were not explicitly designed for, a pattern central to ongoing debates about the field's trajectory toward more general intelligence.

## Recent developments

In the 2020s, [large-language-model](https://www.wikiprompt.org/wiki/large-language-model)s based on the transformer architecture, such as GPT and [hugging-face-transformers](https://www.wikiprompt.org/wiki/hugging-face-transformers) libraries, became mainstream products. Deep learning also expanded into generative models like GANs and diffusion models for image synthesis, and neural radiance fields for 3D scene reconstruction. Advances in [representation-learning](https://www.wikiprompt.org/wiki/representation-learning) and [unsupervised-learning](https://www.wikiprompt.org/wiki/unsupervised-learning) continue to reduce the dependence on labeled data. The field remains a focus of intense research and public debate, with ongoing efforts to improve interpretability, efficiency, and alignment with human values.

---
Source: https://www.wikiprompt.org/wiki/deep-learning
License: CC BY-SA 4.0 (https://creativecommons.org/licenses/by-sa/4.0/)
Last updated: 2026-11-09T12:24:35.012392+00:00