Parti is an autoregressive text-to-image model developed by Google DeepMind. It generates images from textual descriptions by treating image generation as a sequence-to-sequence problem, similar to how large language models generate text. Parti was introduced in 2022 and is notable for its ability to produce high-fidelity, photorealistic images from complex prompts, including scenes with multiple objects and specific attributes.
The model operates by first converting an input text prompt into a sequence of tokens using an encoder, and then autoregressively predicting a sequence of image tokens. These image tokens are derived from a discrete visual codebook, which is learned separately. This approach allows Parti to scale effectively with model size and training data, leading to improvements in image quality and semantic alignment.
Architecture
Parti's architecture is based on the Transformer model. It uses an encoder-decoder structure where the encoder processes the text prompt, and the decoder generates the image token sequence. The visual tokens are obtained from a pre-trained Vision Transformer (ViT) model, which acts as a tokenizer. The decoder is trained to predict the next image token given the text and previously generated tokens, a process analogous to machine learning sequence modeling.
The model was trained in various sizes, ranging from 350 million to 20 billion parameters. The largest variant, Parti-20B, demonstrated state-of-the-art performance on benchmarks like MS-COCO and Localized Narratives, achieving high FID (Fréchet Inception Distance) scores, which measure the similarity between generated and real image distributions.
Training and Data
Parti was trained on a large dataset of image-text pairs, which included publicly available web data and proprietary datasets. The training process involved two stages: first, learning the visual codebook using a ViT-based tokenizer, and second, training the autoregressive transformer to predict image tokens from text. The model was optimized using a standard cross-entropy loss over the discrete image tokens.
Google reported that scaling the model size and training data led to consistent improvements in image quality, particularly for complex prompts requiring compositional reasoning. The model also showed an ability to handle prompts with multiple objects, spatial relationships, and stylistic attributes.
Capabilities and Limitations
Parti excels at generating images with high fidelity and semantic accuracy. It can produce images from prompts that describe specific scenes, such as "a photo of a cat wearing a hat" or "a painting of a futuristic city at sunset." The model also supports text rendering, which is a challenging task for many image generation models. However, Parti has limitations, including occasional errors in fine-grained details, such as incorrect object counts or minor attribute mismatches. It also requires substantial computational resources for training and inference.
Comparison with Other Models
Parti was developed concurrently with other text-to-image models like DALL-E 2 and Imagen. While Imagen uses a diffusion-based approach, Parti is autoregressive, which allows it to leverage advances in sequence modeling. In evaluations, Parti-20B achieved competitive or superior results on benchmarks like MS-COCO compared to DALL-E 2 and Imagen, particularly in terms of FID scores and human evaluations. However, diffusion models like Imagen were noted for their strong performance on certain aesthetic qualities, while Parti excelled in semantic alignment and complex reasoning.
Impact and Legacy
Parti contributed to the rapid advancement of generative AI in image synthesis. Its autoregressive approach influenced subsequent research in unified multimodal models, where text and images are processed in a shared token space. Parti's codebook and scaling insights were later incorporated into other Google models, such as the Gemini family, which integrate text, image, and audio generation. The model also highlighted the importance of scaling laws in vision-language tasks, aligning with findings in deep learning research.