An autoencoder is a type of Neural network trained to reproduce its own input after passing it through a narrow bottleneck layer. It consists of two parts: an encoder that compresses the input into a compact internal code, and a decoder that reconstructs the original input from that code. Because the network can only pass a small amount of information through the bottleneck, it is forced to learn which features of the data matter most, producing a Latent space representation that is smaller and often more meaningful than the raw input.
History and variants
Autoencoders date to the 1980s connectionist literature but gained modern relevance as a form of Unsupervised learning, since they require no labels: the input itself serves as the training target, an early example of what is now called Self-supervised learning. The 2013 introduction of the variational autoencoder (VAE) added a probabilistic twist, constraining the latent space to approximate a smooth, known distribution rather than an arbitrary shape. This made it possible to generate new samples by drawing random points from that distribution and decoding them, turning autoencoders from a compression tool into a generative one. Other variants include denoising autoencoders, trained to recover a clean input from a corrupted version, and sparse autoencoders, which encourage most latent units to stay near zero, a property later reused in Mechanistic interpretability research to isolate individual features inside large language models.
Applications
Beyond direct generation, autoencoders are widely used for dimensionality reduction and anomaly detection, since inputs the model reconstructs poorly are likely to be unusual relative to the training distribution. In image generation, the encoder-decoder pair from an autoencoder is a key component of latent Diffusion models such as Stable Diffusion: an image is compressed into a small latent grid, the diffusion process runs in that compressed space, and the decoder expands the result back into a full-resolution image. This division of labor is a major reason modern Text-to-image generation systems can run on consumer hardware. Autoencoders also appear in recommendation systems, where they help reconstruct sparse user-item interaction data, and in Speech recognition and audio pipelines as a way to compress waveforms before further processing.
Relationship to other architectures
Autoencoders are related to but distinct from Generative adversarial networks: a GAN's generator also maps a latent vector to an output, but it is trained adversarially against a discriminator rather than to minimize reconstruction error, which tends to produce sharper but less controllable outputs. Transformer-based systems have largely displaced simple autoencoders for large-scale Natural language processing, though the encoder-decoder pattern itself, formalized earlier in Seq2seq models, remains a conceptual ancestor of both autoencoders and the Transformer (architecture) architecture. Masked-prediction pretraining objectives used in models such as BERT can also be viewed as a denoising-autoencoder task applied to text.
Limitations
Standard autoencoders optimize only for reconstruction, which does not guarantee a latent space that is smooth, disentangled, or useful for downstream tasks; variational and other regularized variants address this partially but usually trade off reconstruction sharpness for better-structured latent spaces. As of the mid-2020s, autoencoders survive mainly as a component inside larger generative pipelines rather than as a standalone modeling approach.