# Something-Something Something-Something is a large-scale video dataset for fine-grained action recognition, containing over 100,000 videos of humans performing basic actions with everyday objects, designed to test models' understanding of temporal dynamics and causal relationships. Something-Something is a large-scale video dataset for fine-grained action recognition. It was introduced in 2017 by a team of researchers at the University of Antwerp and the Flemish Supercomputer Centre (VSC), in collaboration with the company Something, Inc. The dataset was created to address a key limitation in earlier action recognition datasets: the tendency to rely on static visual cues, such as objects and scenes, rather than on the temporal dynamics of the action itself. Something-Something focuses on basic, everyday actions that are defined by their motion and the causal relationship between the actor and the objects involved, making it a challenging benchmark for models that must understand how actions unfold over time. The dataset consists of over 100,000 videos, each showing a person performing one of 174 distinct action categories. These categories are deliberately simple and abstract, such as 'pushing something from left to right', 'picking something up', or 'turning something upside down'. The actions are performed with a wide variety of everyday objects, including cups, books, and tools, which are not fixed across videos. This design forces models to focus on the motion and interaction patterns rather than on object identity. The videos were crowd-sourced from a global pool of contributors, resulting in a diverse range of backgrounds, lighting conditions, and camera perspectives. Each video is a short clip, typically lasting 2 to 5 seconds, and is labeled with a single action category. ## Motivation and Design The creation of Something-Something was motivated by the observation that many popular action recognition datasets, such as UCF-101 and HMDB-51, contained videos where the action could be inferred from a single frame. For example, a video labeled 'playing guitar' could be recognized simply by the presence of a guitar. This allowed models to achieve high accuracy by exploiting static appearance biases, without truly understanding motion. Something-Something was designed to eliminate these shortcuts. In the dataset, the same object can appear in many different actions, and the same action can be performed on many different objects. Therefore, a model must analyze the temporal sequence of frames to correctly identify the action, making it a more faithful test of temporal reasoning. ## Benchmark and Evaluation The dataset is commonly used to evaluate the performance of video understanding models, particularly those based on [deep learning](https://www.wikiprompt.org/wiki/deep-learning) and [neural networks](https://www.wikiprompt.org/wiki/neural-network). Standard evaluation protocols involve training on a provided training split and reporting top-1 and top-5 accuracy on a validation set. Since its release, Something-Something has become a standard benchmark in the field, alongside other datasets like Kinetics and Moments in Time. It is particularly known for being difficult: even state-of-the-art models, including those using [transformers](https://www.wikiprompt.org/wiki/transformer) and [residual networks](https://www.wikiprompt.org/wiki/residual-network), achieve significantly lower accuracy on this dataset compared to appearance-heavy datasets. This has spurred research into architectures that better capture temporal dynamics, such as 3D convolutional networks and video transformers. ## Impact and Limitations Something-Something has had a significant impact on the field of [artificial intelligence](https://www.wikiprompt.org/wiki/artificial-intelligence) and [machine learning](https://www.wikiprompt.org/wiki/machine-learning), pushing the community toward more robust temporal modeling. It has also been used in research on [data augmentation](https://www.wikiprompt.org/wiki/data-augmentation) and [curriculum learning](https://www.wikiprompt.org/wiki/curriculum-learning) for video. However, the dataset is not without limitations. The action categories are highly abstract, and the crowd-sourced nature of the videos introduces label noise and variability. Some researchers have noted that the actions are often performed in a scripted manner, which can differ from naturalistic human behavior. Additionally, the dataset is relatively small compared to more recent large-scale video datasets, which can limit the effectiveness of [large language models](https://www.wikiprompt.org/wiki/large-language-model) and other data-hungry approaches when used directly for pretraining. ## Related Work and Extensions Several follow-up datasets have been inspired by Something-Something, including Something-Else, which focuses on compositional action recognition, and the Something-Something V2, which is a larger version with more videos and improved annotations. These datasets continue to be used in conjunction with advances in [AI](https://www.wikiprompt.org/wiki/artificial-intelligence) hardware, such as [AWS Trainium](https://www.wikiprompt.org/wiki/aws-trainium) and [Google Cloud](https://www.wikiprompt.org/wiki/google-cloud) TPUs, to train increasingly complex models. The challenge posed by Something-Something remains an active area of research, with ongoing efforts to develop architectures that can reason about the causal and temporal structure of actions, a capability that is essential for applications in robotics, autonomous driving, and human-computer interaction. --- Source: https://www.wikiprompt.org/wiki/something-something License: CC BY-SA 4.0 (https://creativecommons.org/licenses/by-sa/4.0/) Last updated: 2026-09-13T03:53:03.791748+00:00