The Fréchet inception distance (FID) is a metric used to assess the quality of images created by a generative model, such as a generative adversarial network (GAN) or a diffusion model. It measures the similarity between the distribution of features extracted from a set of real images and the distribution of features from a set of generated images. The FID was introduced in 2017 and has become a standard benchmark for evaluating image generation models, including high-resolution models like StyleGAN and various diffusion models.
The FID is inspired by the earlier inception score (IS), which evaluates only the diversity and clarity of generated images. However, the FID provides a more comprehensive comparison by also considering how well the generated images match the real data distribution. A lower FID indicates that the generated images are more similar to real images in terms of feature statistics, while a higher FID suggests poorer quality or less diversity.
Overview
The purpose of the FID is to measure the diversity and fidelity of images produced by a generative model relative to a reference dataset, such as ImageNet or COCO-2014. Using a large and diverse reference set is crucial because it represents the full range of images the model aims to generate. Generative models produce novel images that share features with the training set but are not identical to any single image, so pixel-by-pixel comparisons (e.g., using the L2 norm) are inadequate.
Instead, the FID models the two sets of images as if they were drawn from two multidimensional Gaussian distributions. The features are extracted from the penultimate pooling layer of an Inception v3 network, which captures high-level semantic information about objects and scenes. The distance between these two Gaussian distributions is computed using the Fréchet distance, also known as the 2-Wasserstein distance.
Mathematical Definition
For two probability distributions \(\mu\) and \(\nu\) over \(\mathbb{R}^n\) with finite means and covariances, the Fréchet distance is defined as:
\[ d_F(\mu, \nu) = \left( \inf_{\gamma \in \Gamma(\mu, \nu)} \int_{\mathbb{R}^n \times \mathbb{R}^n} \|x - y\|^2 \, d\gamma(x, y) \right)^{1/2} \]
where \(\Gamma(\mu, \nu)\) is the set of all couplings (joint distributions) with marginals \(\mu\) and \(\nu\). When both distributions are Gaussian, this simplifies to:
\[ d_F(\mathcal{N}(\mu, \Sigma), \mathcal{N}(\mu', \Sigma')) = \|\mu - \mu'\|_2^2 + \operatorname{tr}\left( \Sigma + \Sigma' - 2 (\Sigma \Sigma')^{1/2} \right) \]
In practice, the FID is computed by extracting 2048-dimensional feature vectors from the last pooling layer of Inception v3 for both the real and generated image sets. The sample mean and covariance of these features are then plugged into the formula above. A perfect model would have identical feature distributions, yielding an FID of 0.
Applications and Limitations
The FID is widely used to compare generative models, tune hyperparameters, and monitor training progress. It correlates well with human perception of image quality in many cases, but it has limitations. The metric relies on the Inception network's features, which may not capture all aspects of visual quality, and it assumes Gaussian distributions, which may not hold for complex real-world data. Additionally, the FID can be sensitive to the number of samples used for evaluation, so it is typically computed on a large set (e.g., 10,000 or more images) for stability.
Related Metrics
The FID is part of a family of distribution-based metrics for generative models. The inception score focuses on the entropy of class predictions for generated images, while the Kernel Inception Distance (KID) uses a maximum mean discrepancy approach. More recent work has explored using CLIP embeddings instead of Inception features to better align with semantic similarity. Despite its limitations, the FID remains a standard tool in the field of generative modeling.
See Also
- generative adversarial network
- diffusion model
- inception score
- StyleGAN
- ImageNet
- COCO
- CLIP
- Wasserstein distance