# ImageNet-Sketch

ImageNet-Sketch is a dataset of 50,000 black-and-white sketch images across 1,000 ImageNet classes, designed to evaluate model robustness under domain shift from natural photos to abstract line drawings.

ImageNet-Sketch is a benchmark dataset introduced in 2019 to test how well image classification models generalize to a different visual domain. It contains 50,000 grayscale sketch images, with 50 images for each of the 1,000 classes in the original ImageNet dataset. The sketches are collected from Google Image searches for terms like "sketch of [class name]" and are manually filtered to ensure they depict the intended object. Unlike natural photographs, these images are abstract line drawings, often with minimal detail, making them a challenging domain-shift test for models trained on standard photo datasets.

The dataset was created by researchers at the University of California, Berkeley, including Haohan Wang, Songwei Ge, Zachary Lipton, and Eric Xing. It was released alongside a paper titled "Learning Robust Global Representations by Penalizing Local Predictive Power," presented at the 2019 Conference on Neural Information Processing Systems (NeurIPS). The primary motivation was to provide a simple yet effective probe for evaluating a model's robustness to distribution shift, particularly the shift from real-world images to abstract human sketches. ImageNet-Sketch has since become a standard evaluation set in the field of [machine-learning](https://www.wikiprompt.org/wiki/machine-learning) and [deep-learning](https://www.wikiprompt.org/wiki/deep-learning), often used alongside other robustness benchmarks like ImageNet-C and ImageNet-R.

## Construction and Characteristics

The construction process involved querying Google Image search with the pattern "sketch of [class]" for each of the 1,000 ImageNet classes. The returned images were then filtered to remove non-sketch images, such as photos or diagrams, and to ensure each class had at least 50 valid sketches. The final dataset contains exactly 50 images per class, totaling 50,000 images. All images are resized to a consistent resolution, typically 224x224 pixels, to match standard input sizes for [neural-network](https://www.wikiprompt.org/wiki/neural-network) classifiers.

A key characteristic of ImageNet-Sketch is its reliance on human-drawn sketches, which are inherently more abstract and less photorealistic than natural images. These sketches often omit color, texture, and fine-grained details, forcing models to rely on global shape and structural cues. This makes the dataset particularly useful for studying whether models have learned robust, generalizable features or have overfit to superficial statistics in natural images.

## Role in Robustness Evaluation

ImageNet-Sketch is widely used to measure the robustness of image classifiers under domain shift. Models trained on ImageNet typically show a significant drop in accuracy when evaluated on ImageNet-Sketch, often from around 70-80% top-1 accuracy on natural images to 20-30% on sketches. This performance gap highlights the brittleness of many [deep-learning](https://www.wikiprompt.org/wiki/deep-learning) models when faced with out-of-distribution inputs.

The dataset is often included in robustness leaderboards and studies. For example, it is a core component of the ImageNet-C and ImageNet-P corruption benchmarks, though it is distinct in that it represents a natural, human-created domain shift rather than synthetic corruptions. Researchers have used ImageNet-Sketch to evaluate the effectiveness of various robustness techniques, such as [data-augmentation](https://www.wikiprompt.org/wiki/data-augmentation) strategies, adversarial training, and architectural changes like [residual-network](https://www.wikiprompt.org/wiki/residual-network) variants. It has also been used to probe the internal representations of models, revealing that many models rely heavily on texture and local patterns rather than global shape, which sketches disrupt.

## Relationship to Other Benchmarks

ImageNet-Sketch complements other domain-shift datasets like ImageNet-R (which contains artistic renditions such as paintings and cartoons) and ImageNet-A (which contains naturally occurring adversarial examples). While ImageNet-R includes a mix of artistic styles, ImageNet-Sketch focuses specifically on line drawings, providing a cleaner test of shape-based recognition. The dataset is also related to the Sketchy dataset and TU-Berlin sketch dataset, but it is unique in its scale and direct alignment with ImageNet classes.

In the broader context of [artificial-intelligence](https://www.wikiprompt.org/wiki/artificial-intelligence) research, ImageNet-Sketch has been instrumental in highlighting the limitations of current models and driving interest in robust and generalizable learning. It is frequently cited in papers on domain generalization, self-supervised learning, and model interpretability. As of 2025, it remains a standard evaluation tool in computer vision, with many state-of-the-art models reporting their performance on this benchmark.

## Limitations and Criticisms

One limitation of ImageNet-Sketch is that the sketches are collected from web searches, which may introduce biases in style or content. For instance, some classes may have more diverse sketch styles than others, and the filtering process may inadvertently include images that are not pure sketches. Additionally, the dataset is static, so it does not capture evolving sketch styles over time. Despite these issues, its simplicity and ease of use have made it a popular choice for researchers.

Another criticism is that the performance drop on ImageNet-Sketch may not fully reflect real-world domain shifts, as sketches are an extreme abstraction. However, this extremity is precisely what makes the dataset valuable for stress-testing models. It forces researchers to develop methods that go beyond memorizing pixel patterns and instead learn more abstract, transferable representations.

## Usage in Research and Industry

In academic research, ImageNet-Sketch is often used to benchmark new architectures and training methods. For example, studies on vision transformers and convolutional networks frequently report results on this dataset to demonstrate robustness. It is also used in the development of foundation models, where pretraining on diverse data is expected to improve performance on such out-of-distribution sets. In industry, companies working on image recognition systems, such as those in autonomous driving or medical imaging, may use ImageNet-Sketch as a sanity check for model generalization, though it is more common in research settings.

The dataset is publicly available for download, and its usage is governed by the original ImageNet terms, which allow non-commercial research use. It has been cited in thousands of papers, making it one of the most influential robustness benchmarks in the field. As of 2025, it continues to be a relevant and widely used resource, especially as the community focuses on building models that are reliable in diverse and unpredictable environments.

## Future Directions

Looking ahead, researchers are exploring ways to extend ImageNet-Sketch to other domains, such as 3D sketches or animated drawings, and to create more challenging versions with finer-grained classes. There is also interest in using the dataset to train models that are inherently more robust, rather than just evaluating them. For instance, some work has explored incorporating sketch data into training sets to improve shape bias, which could lead to better generalization on other domain shifts. The dataset remains a cornerstone in the ongoing effort to achieve truly robust [artificial-intelligence](https://www.wikiprompt.org/wiki/artificial-intelligence) systems.

---
Source: https://www.wikiprompt.org/wiki/imagenet-s
License: CC BY-SA 4.0 (https://creativecommons.org/licenses/by-sa/4.0/)
Last updated: 2026-09-12T16:27:51.9735+00:00