A neural network is a computational model composed of interconnected layers of simple processing units, loosely inspired by biological neurons, whose connection strengths, or weights, are adjusted through training to approximate a desired function. Neural networks are the core computational structure behind modern Deep learning and, by extension, most contemporary Artificial intelligence systems.
History
The earliest trainable neural model was the Perceptron, introduced in 1958, which could learn to classify simple patterns using a single layer of adjustable weights. Its limitations, documented in a 1969 critique, contributed to a broader loss of research funding known as an AI winter. Interest in multi-layer networks revived in the 1980s with the popularization of Backpropagation, an algorithm for efficiently computing how each weight in a network should change to reduce error, which allowed networks with hidden layers to be trained in practice. Neural networks moved from a niche technique to the dominant approach in AI following the 2012 success of AlexNet, which demonstrated that networks with many layers, trained on large datasets using graphics processing units, could dramatically outperform prior methods on real-world tasks.
Structure
A neural network is organized into layers of units, or neurons, each of which computes a weighted sum of its inputs and passes the result through a nonlinear activation function before sending it to the next layer. Networks with many such layers between input and output are described as "deep," giving deep learning its name. During training, a Loss function measures the gap between the network's output and the desired result, and Gradient descent iteratively adjusts weights to reduce that gap, with backpropagation supplying the necessary gradients efficiently across all layers at once.
Architectural families
Different network architectures suit different kinds of data. Convolutional neural networks use shared, spatially local filters and are well suited to images. Recurrent neural networks, including the LSTM variant, process sequences step by step and were historically used for text and speech before being displaced by the Transformer (architecture) architecture, which processes entire sequences in parallel using attention. Generative architectures such as the Generative adversarial network and Diffusion model apply neural networks to the task of producing new data rather than only classifying or predicting it.
Limitations and interpretability
Neural networks are often criticized as "black boxes": their internal representations are difficult for humans to inspect or explain, even when their outputs are accurate. This has motivated a research subfield sometimes called Mechanistic interpretability, which attempts to reverse-engineer what individual components of a trained network represent. Despite this opacity, neural networks remain the dominant tool across vision, language, audio, and increasingly robotics and scientific applications, largely because their performance continues to improve predictably with more data, more parameters, and more compute.