# ReCoRD ReCoRD is a large-scale reading comprehension benchmark that tests a model's ability to resolve textual ambiguities using commonsense reasoning, requiring systems to predict masked entities in news passages by linking them to candidate entities from the text. ReCoRD (Reading Comprehension with Commonsense Reasoning) is a benchmark dataset for evaluating the reasoning capabilities of artificial intelligence systems, particularly in the domain of natural language understanding. It was introduced in 2018 by researchers at the University of Maryland and the University of Washington. The dataset is designed to measure a model's ability to perform commonsense reasoning while reading, going beyond simple fact retrieval or pattern matching. The core task in ReCoRD involves a passage of text, typically drawn from news articles, where certain entities have been masked out. The model must predict the masked entity from a list of candidate entities that appear elsewhere in the passage. Crucially, the correct answer often requires understanding the context, resolving coreferences, and applying real-world knowledge that is not explicitly stated. For example, if a passage mentions a person and a city, and the masked entity is a location, the model must infer which city is relevant based on the narrative. ReCoRD contains over 120,000 examples, split into training, validation, and test sets. The passages are sourced from news articles, and the queries are generated by masking named entities. The dataset emphasizes that the correct answer is always one of the candidate entities, which are all mentioned in the passage, making it a cloze-style task with a constrained answer space. This design forces models to reason about the relationships between entities and the events described, rather than relying on external knowledge bases. ## Construction and Design The ReCoRD dataset was constructed using a two-stage process. First, news articles were collected from various sources, and named entities were identified using a standard entity recognizer. Then, for each article, a set of queries was created by randomly masking one entity and providing a list of candidate entities that appear in the same article. The candidates include the correct answer and several distractors, which are other entities from the same article. This ensures that the model cannot simply choose the most frequent entity or rely on surface-level statistics. A notable feature of ReCoRD is its emphasis on commonsense reasoning. The queries are designed so that the correct answer is not trivially determined by local context. Instead, the model must integrate information across multiple sentences and infer implicit relationships. For instance, if a passage describes a person winning an award in a specific field, the model must know that the award is typically associated with that field, a piece of commonsense knowledge. The dataset was split into 101,249 training examples, 6,353 validation examples, and 10,000 test examples. The test set is held out, and models are evaluated on it through a submission system. The primary evaluation metric is exact match accuracy, where the model's predicted entity must exactly match the ground truth. ## Evaluation and Performance ReCoRD has become a standard benchmark in the field of natural language processing. Early models, including those based on recurrent neural networks and early transformer architectures, achieved accuracies in the range of 60-70% on the validation set. The introduction of large pre-trained language models, such as BERT and its successors, significantly improved performance, with some models exceeding 90% accuracy by 2020. However, even state-of-the-art models still struggle with certain types of queries that require deep commonsense reasoning, such as those involving temporal reasoning, spatial reasoning, or nuanced social conventions. This has made ReCoRD a valuable tool for diagnosing the limitations of current AI systems. The benchmark has also been used to study the impact of model size, training data, and architectural choices on reasoning ability. ## Relation to Other Benchmarks ReCoRD is part of a broader family of reading comprehension benchmarks that include SQuAD, RACE, and DROP. Unlike SQuAD, which focuses on extracting answer spans from the text, ReCoRD requires selecting from a set of entities, making it a multiple-choice cloze task. Compared to RACE, which is based on English exams, ReCoRD's news domain provides a more diverse and less formalized context. DROP, on the other hand, requires numerical reasoning, whereas ReCoRD emphasizes commonsense inference. ReCoRD is often used in conjunction with other benchmarks to provide a comprehensive evaluation of a model's capabilities. It is particularly relevant for testing the integration of [artificial-intelligence](https://www.wikiprompt.org/wiki/artificial-intelligence) and [machine-learning](https://www.wikiprompt.org/wiki/machine-learning) techniques in understanding natural language. The benchmark has been adopted by research groups at major institutions, including [google-deepmind](https://www.wikiprompt.org/wiki/google-deepmind), [openai](https://www.wikiprompt.org/wiki/openai), and [anthropic](https://www.wikiprompt.org/wiki/anthropic), as part of their model evaluation suites. ## Impact and Future Directions The introduction of ReCoRD has spurred further research into commonsense reasoning in AI. It has highlighted the gap between pattern recognition and true understanding, prompting efforts to develop models that can reason about everyday situations. Subsequent benchmarks, such as CommonsenseQA and Social IQA, have built on the lessons learned from ReCoRD, focusing on more explicit commonsense knowledge. One limitation of ReCoRD is that the candidate entities are all from the same passage, which can sometimes make the task easier if the model learns to exploit statistical cues. Researchers have proposed variants that introduce external knowledge or require multi-hop reasoning. Despite these challenges, ReCoRD remains a widely cited and used benchmark, and it continues to inform the development of more robust and interpretable AI systems. As of the early 2020s, ReCoRD is still considered a relevant benchmark, though newer models often achieve near-perfect scores. Its legacy lies in demonstrating the importance of commonsense reasoning in natural language understanding and providing a concrete task for measuring progress in this area. ## See Also - [large-language-model](https://www.wikiprompt.org/wiki/large-language-model) - [transformer](https://www.wikiprompt.org/wiki/transformer) - [deep-learning](https://www.wikiprompt.org/wiki/deep-learning) - [neural-network](https://www.wikiprompt.org/wiki/neural-network) --- Source: https://www.wikiprompt.org/wiki/record License: CC BY-SA 4.0 (https://creativecommons.org/licenses/by-sa/4.0/) Last updated: 2026-09-13T03:53:57.225762+00:00