# ImageNet Roulette

ImageNet Roulette is a 2019 art project by Trevor Paglen and Kate Crawford that exposed biases in AI image classification by labeling people with offensive or outdated categories from the ImageNet dataset.

ImageNet Roulette is a web-based art project created by artist Trevor Paglen and AI researcher Kate Crawford in 2019. It allowed users to upload a photo of themselves and receive a label generated by a machine learning model trained on the ImageNet dataset. The project was designed to highlight the biases, stereotypes, and harmful classifications embedded in large-scale visual datasets used to train [artificial-intelligence](https://www.wikiprompt.org/wiki/artificial-intelligence) systems. It gained widespread attention for producing labels such as "slattern," "alcoholic," and "failure" when applied to ordinary people, sparking public debate about the ethics of AI training data.

The project used a neural network trained on a subset of ImageNet that included 2,833 categories related to people, many of which were derived from outdated, offensive, or derogatory terms. Paglen and Crawford sourced these categories from WordNet, a lexical database that groups words by semantic relations, which itself contained historical biases. The work was exhibited at the 2019 Berlin Transmediale festival and later became an online sensation, with millions of users testing it on their own photos. The project was taken offline in late 2019 after the artists received criticism for the emotional harm it caused, particularly to marginalized groups who were labeled with racial or gender-based slurs.

## Background and Motivation

Paglen and Crawford created ImageNet Roulette as part of a broader research effort into the politics of visual datasets. They argued that datasets like ImageNet, which contains over 14 million images across 20,000 categories, are not neutral repositories but reflect the cultural and historical assumptions of their creators. The project aimed to make visible the "invisible" labor and judgment involved in dataset construction, where annotators and curators make subjective decisions about how to categorize people and objects.

The artists drew on Crawford's earlier work on the social implications of AI, including her 2017 paper "The Trouble with Bias," which examined how bias in AI systems is often treated as a technical problem rather than a structural one. Paglen's background in photography and surveillance studies informed the project's focus on how classification systems shape perception and power. Together, they published an accompanying essay titled "Excavating AI: The Politics of Images in Machine Learning Training Sets," which detailed the historical origins of many ImageNet categories.

## Technical Implementation

The project employed a convolutional neural network (CNN) trained on the ImageNet dataset, specifically using the architecture known as [ResNet](https://www.wikiprompt.org/wiki/residual-network)-50. The model was fine-tuned on a subset of 2,833 person-related categories, which included both benign labels like "bride" and "soldier" and problematic ones like "psychopath" and "criminal." The training process used standard [machine-learning](https://www.wikiprompt.org/wiki/machine-learning) techniques, including [data-augmentation](https://www.wikiprompt.org/wiki/data-augmentation) and [batch-normalization](https://www.wikiprompt.org/wiki/batch-normalization), to improve accuracy on the classification task.

When a user uploaded a photo, the system would run inference and return the top predicted label with a confidence score. The interface displayed the label alongside the user's image, often with a caption explaining the category's origin. For example, the category "slattern" was traced back to a 19th-century English dictionary definition of a dirty or untidy woman. The project's code was made publicly available on GitHub, allowing others to replicate or modify the system.

## Public Reception and Impact

ImageNet Roulette received extensive media coverage from outlets like The New York Times, The Guardian, and Wired. Many users shared their results on social media, with some expressing amusement and others reporting distress. The project became a case study in AI ethics courses and was cited in academic papers on algorithmic fairness and dataset documentation. It also influenced subsequent efforts to audit and revise large datasets, such as the ImageNet team's later removal of 600,000 images and the reclassification of certain categories.

The project's impact extended beyond academia. It contributed to a growing awareness of the need for better [data governance](https://www.wikiprompt.org/wiki/data-augmentation) practices in AI development. In response to the controversy, some tech companies, including [Google Cloud](https://www.wikiprompt.org/wiki/google-cloud) and [Amazon Web Services](https://www.wikiprompt.org/wiki/amazon-web-services), announced stricter guidelines for labeling and moderating training data. The project also inspired other artists and researchers to create similar "bias probes," such as the 2020 "Pulse" project by the University of Washington, which highlighted racial bias in face generation models.

## Ethical and Artistic Significance

ImageNet Roulette is often discussed as a landmark example of "critical AI" art, where the goal is not to build functional systems but to expose their underlying assumptions. The project demonstrated that machine learning models are not objective tools but encode the values and prejudices of their creators. By turning the classification process back on the user, it forced individuals to confront how AI might perceive them, creating a visceral experience of algorithmic othering.

The artists also raised questions about consent and harm in AI research. While the project was intended as a critique, it inadvertently caused real emotional distress to users, leading to debates about the responsibility of artists and researchers when exposing sensitive issues. Paglen and Crawford defended the work as necessary provocation, but they also acknowledged the need for more careful consideration of potential harms in future projects. The project remains a reference point in discussions about the ethics of dataset curation and the social impact of [deep-learning](https://www.wikiprompt.org/wiki/deep-learning) systems.

---
Source: https://www.wikiprompt.org/wiki/imagenet-roulette
License: CC BY-SA 4.0 (https://creativecommons.org/licenses/by-sa/4.0/)
Last updated: 2026-09-12T16:26:34.958447+00:00