# LeNet LeNet is a series of convolutional neural network architectures developed at AT&T Bell Laboratories from 1988 to 1998, primarily by Yann LeCun, for handwritten digit recognition. The most famous version, LeNet-5, was historically important in deep learning development. LeNet is a series of convolutional neural network (CNN) architectures created by a research group at AT&T Bell Laboratories between 1988 and 1998, centered around Yann LeCun. These networks were designed for reading small grayscale images of handwritten digits and letters, and were used in ATMs for reading cheques. Convolutional neural networks are a kind of feed-forward neural network whose artificial neurons can respond to a part of the surrounding cells in the coverage range and perform well in large-scale image processing. LeNet-5 was one of the earliest convolutional neural networks and was historically important during the development of [deep learning](https://www.wikiprompt.org/wiki/deep-learning). In general, when LeNet is referred to without a number, it refers to the 1998 version, LeNet-5, which is the most well-known version. ## Development history In October 1988, LeCun joined the Adaptive Systems Research Department at AT&T Bell Laboratories in Holmdel, New Jersey, headed by Lawrence D. Jackel. In late November 1988, John S. Denker et al. published a neural network design for recognizing handwritten zip codes, but its first few layers of connections were hand-tuned. In 1989, Yann LeCun et al. at Bell Labs first applied the backpropagation algorithm to practical applications, believing that the ability to learn network generalization could be greatly enhanced by providing constraints from the task's domain. They combined a convolutional neural network trained by backpropagation algorithms to read handwritten numbers and successfully applied it in identifying handwritten zip code numbers provided by the US Postal Service. This was the prototype of what later came to be called LeNet-1. In the same year, LeCun described a small handwritten digit recognition problem in another paper, showing that even though the problem is linearly separable, single-layer networks exhibited poor generalization capabilities. When using shift-invariant feature detectors on a multi-layered, constrained network, the model could perform very well. He believed that these results proved that minimizing the number of free parameters in the neural network could enhance its generalization ability. In 1989, their paper introducing LeNet-1 described the application of backpropagation networks in handwritten digit recognition again. They performed minimal preprocessing on the data, and the model was carefully designed for the task and highly constrained. The input data consisted of images, each containing a number, and the test results on the postal code digital data provided by the US Postal Service showed that the model had an error rate of only 1% and a rejection rate of about 9%. Their research continued for the next four years, and in 1994 the MNIST database was developed, for which LeNet-1 was too small, hence a new LeNet-4 was trained on it. A year later the AT&T Bell Labs collective reviewed various methods on handwritten character recognition in a paper, using standard handwritten digits to identify benchmark tasks. These models were compared and the results showed that the latest network outperformed other models. By 1998 Yann LeCun, Leon Bottou, Yoshua Bengio, and Patrick Haffner were able to provide examples of practical applications of neural networks, such as two systems for recognizing handwritten characters online and models that could read millions of checks per day, which includes a description of LeNet-5. The research achieved great success and aroused the interest of scholars in the study of neural networks. While the architecture of the best performing neural networks today are not the same as that of LeNet, the network was the starting point for a large number of neural network architectures, and also brought inspiration to the field. ## Architecture LeNet has several common motifs of modern convolutional neural networks, such as convolutional layers, pooling layers, and fully connected layers. Every convolutional layer includes three parts: convolution, pooling, and nonlinear activation functions. It uses convolution to extract spatial features (convolution was called receptive fields originally), subsampling average pooling, tanh activation functions, fully connected layers in the final layers for classification, and sparse connections between layers to reduce the complexity of computation. In 1989, LeCun et al. published a report containing "Net-1" to "Net-5". There were many subsequent refinements up to 1998, and the naming is inconsistent. Generally, when people speak of "LeNet" they refer to the 1998 LeNet, also known as "LeNet-5". LeNet-1, 4, and 5 have been referred to, but it is unclear what LeNet-2 and LeNet-3 might refer to. ### 1988 Net The first neural network published by the LeCun research group was in 1988. It was a hybrid approach. The first stage scaled, deskewed, and skeletonized the input image. The second stage was a convolutional layer with 18 hand-designed kernels. The third stage was a fully connected network with one hidden layer. The dataset was a collection of handwritten digit images extracted from actual U.S. Mail, which was the same dataset used in the famed 1989 report. ### Net-1 to Net-5 Net-1 to Net-5 were published in a 1989 report. The last layer of all of them was fully connected. The original paper does not explain the padding strategy. All cells have an independent bias, including the output cells of convolutional layers. - Net-1: No hidden layer. Fully connected. (16×16) → 10. - Net-2: One hidden fully connected layer with 12 hidden units. (16×16) → 12 → 10. - Net-3: Two hidden convolutional layers. (16×16) → (8×8) → (4×4) → 10. Both are locally connected layers with input shape 3×3 and stride 2. - Net-4: Two hidden layers, the first is a convolution, the second is locally connected. (16×16) → (8×8×2) → (4×4) → 10. The convolution layer has 2 kernels of shape 3×3 and stride 2. The locally connected layer has input shape 3×3 and stride 2. ## Legacy LeNet's influence extends beyond its immediate application. It demonstrated the effectiveness of [backpropagation](https://www.wikiprompt.org/wiki/backpropagation) in training deep networks and introduced key architectural concepts that became standard in later CNNs. The success of LeNet in reading handwritten digits spurred interest in neural networks and laid the groundwork for subsequent developments in [machine learning](https://www.wikiprompt.org/wiki/machine-learning) and [artificial intelligence](https://www.wikiprompt.org/wiki/artificial-intelligence). Its principles are reflected in modern architectures, though they have evolved significantly. The MNIST dataset, developed in 1994 for training LeNet-4, remains a standard benchmark in the field. LeNet's practical deployment in ATMs for cheque reading showcased the real-world viability of neural networks, contributing to the broader adoption of deep learning techniques. --- Source: https://www.wikiprompt.org/wiki/le-net License: CC BY-SA 4.0 (https://creativecommons.org/licenses/by-sa/4.0/) Last updated: 2026-09-07T21:36:09.786665+00:00