# DuoRC

DuoRC is a reading comprehension dataset containing 186,089 question-answer pairs derived from 7,680 movie plot summaries, designed to test machine understanding of long narratives through questions requiring inference across multiple sentences.

DuoRC is a large-scale reading comprehension dataset introduced in 2018 by researchers from the Indian Institute of Technology (IIT) Madras and Carnegie Mellon University. It contains 186,089 question-answer pairs generated from 7,680 movie plot summaries sourced from Wikipedia and IMDb. Unlike many earlier datasets that rely on single-sentence or short-paragraph contexts, DuoRC focuses on long, multi-paragraph narratives, requiring models to synthesize information across multiple sentences and events.

The dataset was constructed using a two-stage process. First, Amazon Mechanical Turk workers wrote questions based on plot summaries from Wikipedia, which are typically neutral and spoiler-free. Then, a separate set of workers answered those questions using corresponding plot summaries from IMDb, which are more detailed and often contain spoilers. This deliberate mismatch between the question-generation context and the answer-finding context forces models to rely on deeper comprehension rather than simple lexical matching, as the answer may not appear verbatim in the source used for question creation.

## Construction and Annotation

Each movie in DuoRC has two distinct plot summaries: one from Wikipedia (average length around 1,000 words) and one from IMDb (average length around 2,000 words). The annotation process involved multiple quality control steps, including filtering out questions that could be answered from a single sentence and ensuring that answers were spans of text present in the IMDb summary. The final dataset contains 96,311 questions for the training split, 12,792 for validation, and 76,986 for testing, with a total of 186,089 question-answer pairs.

Questions in DuoRC are primarily "what," "who," "why," and "how" types, with a significant proportion requiring multi-sentence reasoning. For example, a question might ask why a character acted in a certain way, with the answer requiring information from three or four different paragraphs. This distinguishes DuoRC from datasets like SQuAD, where most answers are located within a single sentence or adjacent sentences.

## Evaluation and Baselines

DuoRC uses standard reading comprehension metrics: Exact Match (EM) and F1 score. When first released, the authors evaluated several baseline models, including traditional feature-based systems and early neural models. The best-performing baseline at the time achieved an EM score of approximately 14% and an F1 score of around 24%, significantly lower than human performance, which was estimated at 78% EM and 86% F1. This large gap highlighted the difficulty of long-context reasoning and spurred further research in the field.

Subsequent work with [transformer](https://www.wikiprompt.org/wiki/transformer)-based models, such as [large language models](https://www.wikiprompt.org/wiki/large-language-model), has improved performance substantially. However, even modern systems struggle with the most complex questions in DuoRC, particularly those requiring cross-paragraph inference and temporal reasoning. The dataset remains a benchmark for evaluating long-document comprehension in [natural language processing](https://www.wikiprompt.org/wiki/natural-language-processing) research.

## Relation to Other Datasets

DuoRC was designed to complement existing reading comprehension datasets. Unlike [SQuAD](https://www.wikiprompt.org/wiki/squad) (which uses short Wikipedia articles) or [RACE](https://www.wikiprompt.org/wiki/race) (which uses exams), DuoRC provides longer contexts and a unique question-answer mismatch. This design choice makes it particularly useful for studying whether models can generalize beyond simple pattern matching. The dataset has been used in research on multi-hop reasoning, machine reading comprehension, and question answering systems.

The creators also released a subset called DuoRC-Paraphrase, which contains questions that are paraphrased versions of original questions, allowing researchers to test robustness to linguistic variation. This subset has been used to evaluate the sensitivity of models to surface-level changes in question phrasing.

## Impact and Limitations

DuoRC has been widely cited in the [artificial intelligence](https://www.wikiprompt.org/wiki/artificial-intelligence) community as a challenging benchmark for long-context understanding. It has influenced the design of subsequent datasets, such as [NarrativeQA](https://www.wikiprompt.org/wiki/narrativeqa), which also uses book and movie plots. However, DuoRC has limitations: the movie plot summaries are copyrighted material, which restricts redistribution, and the dataset is primarily in English, limiting cross-lingual research.

Another limitation is that the questions were generated by crowd workers, which can introduce biases or inconsistencies. Some questions may have multiple valid answers, and the span-based answer format does not capture all types of reasoning. Despite these issues, DuoRC remains a valuable resource for evaluating and improving models that need to process long, narrative text.

## Future Directions

With the rise of [large language models](https://www.wikiprompt.org/wiki/large-language-model) and [generative AI](https://www.wikiprompt.org/wiki/generative-ai), DuoRC has been used to test whether these models can handle long-form reasoning. Recent evaluations show that models like GPT-4 and Claude can achieve much higher scores, but still fall short of human performance on the hardest questions. Researchers continue to use DuoRC to probe weaknesses in current systems, such as hallucination and failure to track character relationships over long passages.

The dataset also serves as a testbed for developing new [attention mechanisms](https://www.wikiprompt.org/wiki/attention-mechanism) and memory-augmented networks designed for long sequences. As [deep learning](https://www.wikiprompt.org/wiki/deep-learning) models become more efficient at processing long contexts, DuoRC will likely remain a standard benchmark for measuring progress in machine comprehension of narratives.

## See Also

- Machine reading comprehension
- Question answering
- [Natural language processing](https://www.wikiprompt.org/wiki/natural-language-processing)
- [Large language models](https://www.wikiprompt.org/wiki/large-language-model)

---
Source: https://www.wikiprompt.org/wiki/duorc
License: CC BY-SA 4.0 (https://creativecommons.org/licenses/by-sa/4.0/)
Last updated: 2026-09-13T03:54:07.293155+00:00