A latent space is a compressed, lower-dimensional space that a neural network learns internally to represent the structure of its training data. Rather than storing raw pixels, words, or audio samples, models that use a latent space map inputs into a set of abstract numerical coordinates, or vectors, chosen so that inputs with similar meaning or appearance end up positioned near one another. The idea is used across Deep learning, from Autoencoders and Generative adversarial networks to modern Diffusion models and Embedding-based search systems.
Origins
The idea traces back to earlier dimensionality-reduction methods in statistics, but it entered mainstream machine learning through the autoencoder, a Neural network trained to compress an input into a bottleneck layer and then reconstruct it. The bottleneck layer's values are the latent representation. Variational autoencoders, introduced in 2013, extended this by forcing the latent space to follow a smooth probability distribution, which made it possible to sample new, plausible outputs by picking random points in that space. A parallel line of work, Word2vec (2013), showed that a latent space trained on text could capture analogical relationships between words, popularizing the idea that direction and distance in a latent space carry meaning.
Role in generative models
Latent space became central to image generation with generative adversarial networks, whose generator learns to map random latent vectors into realistic images. It became even more important with latent diffusion, the technique behind Stable Diffusion and related systems: rather than running the slow denoising process directly on pixels, the model compresses an image into a small latent grid with an autoencoder, performs the diffusion-model steps there, and decodes back to pixels only once. This made high-resolution Text-to-image generation generation dramatically cheaper to run. Multimodal systems such as CLIP learn a shared latent space for images and text, so that a caption and its matching picture land near each other, which is what lets a diffusion model be guided by a text prompt in the first place.
Interpolation, arithmetic, and search
Because nearby points in a well-formed latent space tend to be semantically similar, practitioners can interpolate smoothly between two points to morph one output into another, or perform vector arithmetic, a famous early example being "king minus man plus woman equals queen" in word embeddings. This same property underlies Semantic search: documents and queries are embedded into a shared latent space, and a Vector database finds nearby vectors rather than matching exact keywords. In image models, editing a latent vector along a discovered direction can change an attribute of the output, such as lighting or pose, without retraining the model.
Limitations
Latent spaces are learned, not designed, so their structure is not guaranteed to be interpretable or evenly organized; regions can be entangled, meaning a single dimension affects several visible attributes at once. Work in Mechanistic interpretability has tried to make latent representations more transparent, with mixed success, and the geometry of a latent space can shift substantially between model versions, complicating attempts to reuse learned directions across models.