# CLIP

Contrastive Language-Image Pre-training, a neural network released by OpenAI in January 2021 that learns joint representations of images and text, and which became a foundational component of many later text-to-image generation systems.

CLIP (Contrastive Language-Image Pre-training) is a [vision-language model](https://www.wikiprompt.org/wiki/vision-language-model) released by [openai](https://www.wikiprompt.org/wiki/openai) in January 2021. Rather than being trained to classify images into a fixed set of labels, CLIP learns to associate images with free-form text descriptions by training on a large dataset of image-caption pairs collected from the internet, reported at roughly 400 million pairs.

## Method

CLIP consists of two encoders trained jointly: an image encoder and a text encoder, both of which map their respective inputs into a shared [embedding](https://www.wikiprompt.org/wiki/embedding) space. Training uses a contrastive objective: given a batch of image-text pairs, the model is trained to pull the embedding of a matching image and caption close together in that space while pushing non-matching pairs apart. This approach, drawing on earlier contrastive representation-learning ideas, let CLIP learn general-purpose visual concepts directly from the natural-language supervision present in web text and alt-text, rather than requiring hand-labeled classification datasets like [imagenet](https://www.wikiprompt.org/wiki/imagenet).

The resulting model could perform "zero-shot" image classification, a form of [zero-shot-learning](https://www.wikiprompt.org/wiki/zero-shot-learning), by comparing an image's embedding against embeddings of candidate text labels, without ever having been explicitly trained on those categories. This was a marked departure from the dominant [convolutional network](https://www.wikiprompt.org/wiki/convolutional-neural-network)-based image classifiers of the previous decade, which typically required task-specific fine-tuning.

## Role in image generation

CLIP's most consequential impact came not from classification but from its use as a guidance signal for generative image models. Shortly after release, independent researchers and hobbyists combined CLIP with existing image-generation techniques, including early GAN-based methods and later [diffusion models](https://www.wikiprompt.org/wiki/diffusion-model), using CLIP's text-image similarity scores to steer generation toward images that matched a given [prompt](https://www.wikiprompt.org/wiki/prompt). This "CLIP-guided" approach became a foundation of the early open text-to-image art community before purpose-built systems arrived.

CLIP's text encoder was subsequently incorporated directly into major [text-to-image](https://www.wikiprompt.org/wiki/text-to-image) systems, most notably as a conditioning component in [stable-diffusion](https://www.wikiprompt.org/wiki/stable-diffusion), and its underlying architecture and training recipe influenced other multimodal models across the industry, including versions of [dall-e](https://www.wikiprompt.org/wiki/dall-e) and successor vision-language systems. Because CLIP's embeddings capture semantic similarity between images and text, the model has also been used for content-based image search, [semantic search](https://www.wikiprompt.org/wiki/semantic-search) over image collections, and as an automated metric for evaluating how well a generated image matches its prompt.

## Significance and limitations

CLIP is frequently cited as an early and influential example of scaling contrastive pretraining on noisy, web-scale data rather than relying on curated labels, a strategy later echoed across [multimodal AI](https://www.wikiprompt.org/wiki/multimodal-ai) research. It has also been studied as a case of [bias](https://www.wikiprompt.org/wiki/algorithmic-bias) inherited from internet training data, with researchers documenting skewed associations along race, gender, and other social categories reflecting biases present in the underlying web-scraped captions. OpenAI released CLIP's weights openly, in contrast to the company's later, mostly closed releases, which helped it become a widely reused building block across the open-source generative AI ecosystem.

---
Source: https://www.wikiprompt.org/wiki/clip
License: CC BY-SA 4.0 (https://creativecommons.org/licenses/by-sa/4.0/)
Last updated: 2026-09-02T20:32:48.749556+00:00