A variational autoencoder (VAE) is an artificial neural network architecture introduced by Diederik P. Kingma and Max Welling in 2013. It belongs to the families of probabilistic graphical models and variational Bayesian methods. In addition to being an autoencoder architecture, a VAE can be studied within the mathematical formulation of variational Bayesian methods, connecting a neural encoder network to its decoder through a probabilistic latent space, such as a multivariate Gaussian distribution, that corresponds to the parameters of a variational distribution.
The encoder maps each point from a large complex dataset, such as an image, into a distribution within the latent space rather than to a single point. The decoder performs the opposite function, mapping from the latent space to the input space according to a distribution, although noise is rarely added during decoding in practice. By mapping a point to a distribution instead of a single point, the network can avoid overfitting the training data. Both networks are typically trained together using the reparameterization trick, though the variance of the noise model can be learned separately. Although initially designed for unsupervised learning, the model has proven effective for semi-supervised and supervised learning.
Architecture and operation
A VAE is a generative model with a prior and noise distribution. Such models are often trained using the expectation-maximization meta-algorithm, as with probabilistic PCA or sparse coding. This scheme optimizes a lower bound of the data likelihood, which is usually computationally intractable, and requires discovering q-distributions, or variational posteriors. These q-distributions are normally parameterized for each individual data point in a separate optimization process. However, VAEs use a neural network as an amortized approach to jointly optimize across data points, reusing the same parameters for multiple data points and achieving massive memory savings. The first neural network, called the encoder, takes data points as input and outputs parameters for the variational distribution, mapping from a known input space to the low-dimensional latent space.
The decoder is the second neural network, mapping from the latent space to the input space, for example as the means of the noise distribution. It is possible to use another neural network to map to the variance, but this can be omitted for simplicity; in such a case, the variance can be optimized with gradient descent.
To optimize the model, two terms are needed: the reconstruction error and the Kullback–Leibler divergence (KL-D). Both terms derive from the free energy expression of the probabilistic model and differ depending on the noise distribution and the assumed prior of the data, referred to as the p-distribution. For example, a standard VAE task such as IMAGENET typically assumes Gaussian noise, while tasks like binarized MNIST require Bernoulli noise. The KL-D term maximizes the probability mass of the q-distribution that overlaps with the p-distribution, which can result in mode-seeking behavior. The reconstruction term is the remainder of the free energy expression and requires a sampling approximation to compute its expectation value. More recent approaches replace KL-D with various statistical distances.
Formulation
From the perspective of probabilistic modeling, one wants to maximize the likelihood of the data x by their chosen parameterized probability distribution p_θ(x) = p(x|θ). This distribution is usually chosen to be a Gaussian N(x|μ, σ), parameterized by μ and σ, and as a member of the exponential family it is easy to work with as a noise distribution. Simple distributions are easy to maximize, but distributions with a prior over the latents z result in intractable integrals. To find p_θ(x), one marginalizes over z:
p_θ(x) = ∫_z p_θ(x, z) dz,
where p_θ(x, z) is the joint distribution under p_θ of the observable data x and its latent representation z. By the chain rule, this can be rewritten as p_θ(x) = ∫_z p_θ(x|z) p_θ(z) dz.
This integral is typically intractable, so the VAE introduces an encoder network to approximate the posterior p_θ(z|x) with a variational distribution q_φ(z|x). The training objective is the evidence lower bound (ELBO), which combines the reconstruction term and the KL divergence, and is optimized via stochastic gradient descent using the reparameterization trick.
Training and the reparameterization trick
The reparameterization trick is a key technique for training VAEs. It allows backpropagation through stochastic sampling by expressing the latent variable z as a deterministic function of the input and a noise variable. For a Gaussian latent space, z is sampled as z = μ + σ ⊙ ε, where ε is drawn from a standard normal distribution. This makes the gradient of the ELBO computable with respect to the encoder and decoder parameters.
The encoder outputs the mean μ and variance σ of the variational distribution, and the decoder reconstructs the input from the sampled z. The reconstruction error measures how well the decoder reconstructs the input, while the KL divergence regularizes the latent space to match the prior, typically a standard normal distribution. This balance encourages a smooth, continuous latent space that can be sampled for generation.
Applications and variants
VAEs have been widely applied in Generative AI for image generation, anomaly detection, and representation learning. They are also used in semi-supervised learning, where the latent space can capture meaningful features for classification. Variants include the beta-VAE, which weights the KL term to encourage disentangled representations, and the VQ-VAE, which uses discrete latent codes. Statistical distance VAE variants replace KL-D with other distances to improve robustness or sample quality.
Compared to other generative models like generative adversarial networks (GANs), VAEs offer stable training and a principled probabilistic framework, but often produce blurrier samples. They are also related to Deep learning architectures and are a foundational topic in Machine learning. The model has been extended in numerous ways, including conditional VAEs and hierarchical VAEs, and remains an active area of research in Artificial intelligence.