DALL-E is a series of text-to-image models developed by OpenAI using deep learning methodologies to generate digital images from natural language descriptions known as prompts. The first version was announced in January 2021, followed by DALL-E 2 in 2022 and DALL-E 3 in 2023. The name is a portmanteau of the Pixar robot character WALL-E and the Spanish surrealist artist Salvador DalÃ.
The models are part of the broader field of generative artificial intelligence, which focuses on creating new content such as images, text, or audio. DALL-E's capabilities have influenced subsequent developments in text-to-image generation and have been integrated into various commercial products.
History and background
OpenAI revealed DALL-E in a blog post on 5 January 2021, using a modified version of its GPT-3 model to generate images. On 6 April 2022, the company announced DALL-E 2, a successor designed to produce more realistic images at higher resolutions and to combine concepts, attributes, and styles. Access to DALL-E 2 was initially restricted to pre-selected users for a research preview due to ethical and safety concerns. On 20 July 2022, the model entered a beta phase with invitations sent to 1 million waitlisted individuals, allowing a certain number of free images per month with options to purchase more. The waitlist requirement was removed on 28 September 2022, opening the model to everyone.
In September 2023, OpenAI announced DALL-E 3, which could understand significantly more nuance and detail than previous iterations. It was released natively into ChatGPT for ChatGPT Plus and ChatGPT Enterprise customers in October 2023, with availability via OpenAI's API and Labs platform in early November of that year. Microsoft implemented DALL-E 3 in Bing's Image Creator tool and planned to use it in its Designer app, with Microsoft Copilot running on DALL-E 3. In March 2025, DALL-E 3 was replaced in ChatGPT by GPT Image's native image-generation capabilities.
In February 2024, OpenAI began adding watermarks to DALL-E generated images, containing metadata in the C2PA (Coalition for Content Provenance and Authenticity) standard promoted by the Content Authenticity Initiative.
Technology
The first generative pre-trained transformer (GPT) model was developed by OpenAI in 2018 using a Transformer architecture. It was scaled up to GPT-2 in 2019 and GPT-3 in 2020, with 175 billion parameters. DALL-E builds on this foundation.
DALL-E
The original DALL-E has three components: a discrete variational autoencoder (VAE), an autoregressive decoder-only Transformer model with 12 billion parameters similar to GPT-3, and a CLIP pair of image and text encoders. The discrete VAE converts an image into a sequence of tokens and back, since the Transformer does not process image data directly. The input is a sequence of tokenised image captions followed by tokenised image patches. Captions are in English, tokenised by byte pair encoding with a vocabulary size of 16,384, and can be up to 256 tokens long. Each image is 256 by 256 RGB, divided into 32 by 32 patches of 4 by 4 pixels, with each patch converted by the VAE to a token from a vocabulary of 8,192.
DALL-E was developed alongside CLIP (Contrastive Language-Image Pre-training), a separate model based on contrastive learning trained on 400 million pairs of images and text captions scraped from the Internet. CLIP ranks DALL-E's output by predicting which caption from a list of 32,768 randomly selected captions is most appropriate for an image. A trained CLIP pair filters a larger initial list of images to select the one closest to the text prompt.
DALL-E 2
DALL-E 2 uses 3.5 billion parameters, fewer than its predecessor. Instead of an autoregressive Transformer, it uses a diffusion model conditioned on CLIP image embeddings, which are generated from CLIP text embeddings by a prior model during inference. This architecture is the same as that of Stable Diffusion, released a few months later.
DALL-E 3
A technical report for DALL-E 3 was written, but it does not include training or implementation details, focusing instead on improved prompt following capabilities.
Capabilities
DALL-E can generate imagery in multiple styles, including photorealistic imagery, paintings, and emoji. It can manipulate and rearrange objects in images and correctly place design elements in novel compositions without explicit instruction. For example, when asked to draw a daikon radish blowing its nose, sipping a latte, or riding a unicycle, DALL-E often draws the handkerchief, hands, and feet in plausible locations. It can infer appropriate details without specific prompts, such as adding Christmas imagery to prompts commonly associated with the celebration or placing shadows appropriately. DALL-E also exhibits a broad understanding of visual and design trends.
The model can produce images for a wide variety of arbitrary descriptions from various viewpoints with only rare failures. Mark Riedl, an associate professor at the Georgia Tech School of Interactive Computing, found that DALL-E could blend concepts, described as a key element of human creativity. Its visual reasoning ability is sufficient to solve Raven's Matrices, visual tests often administered to humans to measure intelligence.
DALL-E 3 follows complex prompts with more accuracy and detail than its predecessors and can generate more coherent and accurate text. It is integrated into ChatGPT Plus.
Image modification
Given an existing image, DALL-E 2 and DALL-E 3 can produce variations of the image as individual outputs based on the original, as well as edit the image to modify or expand upon it. The inpainting and outpainting abilities of these models use context from an image to fill in missing areas using a medium consistent with the original, following a given prompt. For example, this can be used to insert a new subject into an image or expand an image beyond its original borders. According to OpenAI, outpainting takes into account the image's existing visual elements, including shadows, reflections, and textures.
Impact and adoption
DALL-E has been widely adopted in commercial tools. Microsoft implemented DALL-E 2 in its Designer app and Image Creator tool included in Bing and Microsoft Edge. The API operates on a cost-per-image basis, with prices varying by image resolution, and volume discounts are available to companies working with OpenAI's enterprise team. The model's influence extends to other text-to-image systems and has contributed to discussions about artificial intelligence safety, copyright, and content provenance.