# Charades Charades is a daily activity recognition dataset containing 9,848 videos of 157 action classes, used to train and evaluate computer vision models for human activity understanding in real-world settings. Charades is a large-scale dataset for daily activity recognition, consisting of 9,848 videos collected from 267 users. The dataset was introduced in 2016 by researchers at Carnegie Mellon University and the University of Toronto, and it focuses on recognizing everyday actions performed in home environments. Each video is annotated with multiple action labels, providing a rich resource for training and evaluating machine learning models in the field of computer vision. The dataset was created to address the limitations of earlier activity recognition benchmarks, which often featured staged or scripted actions. Charades captures spontaneous, unscripted activities such as cooking, cleaning, and reading, making it a more realistic benchmark for real-world applications. The videos are accompanied by textual descriptions and action labels, enabling research in both visual recognition and language grounding. ## Dataset Composition Charades contains 9,848 videos with an average duration of about 30 seconds, totaling roughly 82 hours of footage. The dataset includes 157 action classes, covering a wide range of daily activities like "opening a refrigerator," "watching TV," and "using a phone." Each video is annotated with multiple action instances, and the annotations include temporal boundaries, allowing for frame-level action recognition. The videos were collected via Amazon Mechanical Turk, where participants were asked to record themselves performing activities in their own homes. This crowdsourced approach resulted in diverse environments, lighting conditions, and camera angles, which increases the difficulty of the recognition task. The dataset also includes 27,847 video descriptions, which are used for tasks like video captioning and retrieval. ## Evaluation and Benchmarks Charades is commonly used to benchmark action recognition models, particularly those based on deep learning. The standard evaluation metric is mean average precision (mAP) across all action classes, computed at the video level. Researchers often report results on the Charades v1 split, which includes 7,984 training videos and 1,864 test videos. Several notable models have been evaluated on Charades, including two-stream convolutional networks, temporal segment networks, and more recently, transformer-based architectures. The dataset has also been used to study multi-label classification, as each video can contain multiple simultaneous actions. As of 2025, state-of-the-art models achieve mAP scores above 60%, but the dataset remains challenging due to its realistic and noisy nature. ## Applications in Artificial Intelligence Charades has played a significant role in advancing [artificial-intelligence](https://www.wikiprompt.org/wiki/artificial-intelligence) research, particularly in the areas of [machine-learning](https://www.wikiprompt.org/wiki/machine-learning) and [deep-learning](https://www.wikiprompt.org/wiki/deep-learning). It is widely used to train models for video understanding, which is a key component of applications such as surveillance, human-computer interaction, and robotics. The dataset also supports research in [neural-network](https://www.wikiprompt.org/wiki/neural-network) architectures, including [residual-network](https://www.wikiprompt.org/wiki/residual-network) and [transformer](https://www.wikiprompt.org/wiki/transformer) models. Beyond action recognition, Charades has been used for tasks like temporal action localization, where models must identify when actions occur in a video. It has also been employed in multimodal learning, combining visual and textual information. The dataset's annotations enable research in [sequence-to-sequence](https://www.wikiprompt.org/wiki/sequence-to-sequence) models for video captioning, where the goal is to generate natural language descriptions of activities. ## Related Datasets and Impact Charades is part of a broader family of activity recognition datasets, including UCF101, ActivityNet, and Kinetics. Unlike these datasets, which often feature short, trimmed clips, Charades provides longer, untrimmed videos that better reflect real-world conditions. This has made it a valuable resource for developing robust models that can handle temporal dependencies and noisy inputs. The dataset has been cited in hundreds of research papers and has influenced the design of subsequent benchmarks, such as the Charades-Ego dataset, which focuses on egocentric videos. Charades has also been used in challenges at major conferences, including the ActivityNet Large Scale Activity Recognition Challenge, fostering collaboration between academia and industry. ## Limitations and Future Directions Despite its strengths, Charades has limitations. The videos are recorded by non-professionals, leading to variability in quality and occasional annotation errors. The action classes are limited to daily activities, which may not generalize to other domains like sports or medical procedures. Additionally, the dataset is relatively small compared to newer benchmarks, which can limit the training of very large models. Future research may address these limitations by expanding the dataset or combining it with synthetic data. The rise of [generative-ai](https://www.wikiprompt.org/wiki/generative-ai) and [large-language-model](https://www.wikiprompt.org/wiki/large-language-model)s has also opened new avenues for using Charades in multimodal reasoning tasks, where models must understand both visual and textual information. As of 2025, Charades remains a standard benchmark for evaluating progress in video understanding. ## See Also - [machine-learning](https://www.wikiprompt.org/wiki/machine-learning) - [deep-learning](https://www.wikiprompt.org/wiki/deep-learning) - [computer-vision](https://www.wikiprompt.org/wiki/computer-vision) - video-understanding --- Source: https://www.wikiprompt.org/wiki/charades License: CC BY-SA 4.0 (https://creativecommons.org/licenses/by-sa/4.0/) Last updated: 2026-09-13T03:53:09.990037+00:00