DALL-E is a series of text-to-image models developed by OpenAI that generate digital images from natural language descriptions, known as prompts. The first version was announced in January 2021, followed by DALL-E 2 in 2022 and DALL-E 3 in 2023. The name is a portmanteau of the animated robot WALL-E and the surrealist artist Salvador DalÃ.
History and background
OpenAI revealed DALL-E in a blog post on 5 January 2021, using a modified version of its GPT-3 model to generate images. On 6 April 2022, the company announced DALL-E 2, a successor designed to produce more realistic images at higher resolutions and to combine concepts, attributes, and styles. Access was initially restricted to pre-selected users for a research preview due to ethical and safety concerns. On 20 July 2022, DALL-E 2 entered beta, with invitations sent to one million waitlisted individuals; users could generate a limited number of images for free each month and purchase more. On 28 September 2022, the waitlist was removed and the model became available to everyone.
In September 2023, OpenAI announced DALL-E 3, which could understand significantly more nuance and detail than previous iterations. It was released natively into ChatGPT for Plus and Enterprise customers in October 2023, with API and Labs availability in early November. Microsoft integrated DALL-E 3 into Bing's Image Creator and its Designer app, and Microsoft Copilot runs on DALL-E 3. In March 2025, DALL-E 3 was replaced in ChatGPT by GPT Image's native image-generation capabilities.
In February 2024, OpenAI began adding watermarks to DALL-E generated images, embedding metadata in the C2PA (Coalition for Content Provenance and Authenticity) standard promoted by the Content Authenticity Initiative.
Technology
The first generative pre-trained transformer (GPT) model was developed by OpenAI in 2018, using a Transformer (architecture) architecture. It was scaled up to GPT-2 in 2019 and to GPT-3 in 2020, with 175 billion parameters.
DALL-E
DALL-E consists of three components: a discrete variational autoencoder (VAE), an autoregressive decoder-only transformer model with 12 billion parameters similar to GPT-3, and a CLIP (Contrastive Language-Image Pre-training) pair of image and text encoders. The discrete VAE converts an image into a sequence of tokens and back, allowing the transformer to process image data. The transformer receives a sequence of tokenised image captions followed by tokenised image patches. Captions are in English, tokenised by byte pair encoding with a vocabulary size of 16,384, and can be up to 256 tokens long. Each image is 256×256 RGB, divided into 32×32 patches of 4×4 pixels, each converted by the VAE to a token from a vocabulary of 8,192.
CLIP, developed alongside DALL-E, is a separate model based on contrastive learning, trained on 400 million image-text pairs scraped from the internet. It ranks DALL-E's output by predicting which caption from a list of 32,768 randomly selected captions is most appropriate for an image. A trained CLIP pair filters a larger initial list of generated images to select the one closest to the prompt.
DALL-E 2
DALL-E 2 uses 3.5 billion parameters, fewer than its predecessor. Instead of an autoregressive transformer, it uses a diffusion model conditioned on CLIP image embeddings, which are generated from CLIP text embeddings by a prior model during inference. This architecture is similar to that of Stable Diffusion, released a few months later.
DALL-E 3
OpenAI published a technical report for DALL-E 3, but it does not include training or implementation details, focusing instead on improved prompt-following capabilities.
Capabilities
DALL-E can generate imagery in multiple styles, including photorealistic images, paintings, and emoji. It can manipulate and rearrange objects in images and correctly place design elements in novel compositions without explicit instruction. For example, when asked to draw a daikon radish blowing its nose, sipping a latte, or riding a unicycle, DALL-E often draws the handkerchief, hands, and feet in plausible locations. It can infer appropriate details not mentioned in prompts, such as adding Christmas imagery to holiday-related prompts or placing shadows appropriately. DALL-E also exhibits a broad understanding of visual and design trends.
DALL-E can produce images for a wide variety of arbitrary descriptions from various viewpoints, with only rare failures. Mark Riedl, an associate professor at the Georgia Tech School of Interactive Computing, found that DALL-E could blend concepts, which he described as a key element of human creativity. Its visual reasoning ability is sufficient to solve Raven's Matrices, visual tests often used to measure human intelligence.
DALL-E 3 follows complex prompts with more accuracy and detail than its predecessors and can generate more coherent and accurate text. It is integrated into ChatGPT Plus.
Image modification
DALL-E 2 and DALL-E 3 can produce variations of existing images and edit them to modify or expand upon them. Inpainting and outpainting abilities use context from an image to fill in missing areas in a style consistent with the original, following a given prompt. This can insert a new subject into an image or expand it beyond its original borders. According to OpenAI, outpainting takes into account the image's existing visual elements, including shadows, reflections, and textures.