# MRPC

MRPC (Microsoft Research Paraphrase Corpus) is a benchmark dataset of 5,801 sentence pairs from news sources, used to train and evaluate models on paraphrase detection, a core natural language understanding task.

The Microsoft Research Paraphrase Corpus (MRPC) is a standard benchmark dataset in natural language processing (NLP) for the task of paraphrase detection. It consists of 5,801 sentence pairs automatically extracted from thousands of online news sources, with each pair labeled as either semantically equivalent (a paraphrase) or not. The corpus was introduced by researchers at Microsoft Research in 2005 and has since become a widely used evaluation set for [machine-learning](https://www.wikiprompt.org/wiki/machine-learning) models, particularly in the development of [neural-network](https://www.wikiprompt.org/wiki/neural-network) architectures for sentence understanding.

MRPC is notable for its real-world origin: the sentence pairs were harvested from news articles covering the same events, making the paraphrase detection task challenging due to the natural variation in wording, syntax, and detail. The dataset is split into a training set of 4,076 pairs and a test set of 1,725 pairs, with human annotations providing the ground-truth labels. Over the years, MRPC has been incorporated into the broader [GLUE](https://www.wikiprompt.org/wiki/glue) benchmark (General Language Understanding Evaluation), where it serves as a single-sentence and sentence-pair classification task.

## Dataset Construction and Annotation

The MRPC was built by first collecting news articles from a variety of online sources, then using a heuristic to identify candidate sentence pairs that might be paraphrases. The heuristic relied on the similarity of the surrounding context, such as the same event being described in different articles. After this automatic candidate generation, human annotators manually labeled each pair as either "semantically equivalent" or "not equivalent." The annotation process involved multiple judges, and disagreements were resolved through discussion, ensuring a high-quality gold standard. The final dataset contains a roughly balanced distribution of positive and negative examples, with about 67% of pairs labeled as paraphrases.

The original paper describing MRPC, titled "Automatic Evaluation of Paraphrase Quality" (2005), also introduced an automatic metric for paraphrase quality based on the corpus. However, the dataset itself quickly became more influential as a training and testing resource for supervised models. Its moderate size makes it feasible for training classical classifiers and early neural models, while still providing a meaningful challenge for modern [deep-learning](https://www.wikiprompt.org/wiki/deep-learning) systems.

## Role in Model Evaluation

MRPC became a staple in the evaluation of sentence-pair classification models. In the early 2010s, it was used to test recurrent neural networks and convolutional neural networks for semantic similarity. Later, with the advent of [transformer](https://www.wikiprompt.org/wiki/transformer)-based models, MRPC was included in the GLUE benchmark, which standardized the evaluation of general-purpose language models. For example, [BERT](https://www.wikiprompt.org/wiki/bert) (Bidirectional Encoder Representations from Transformers) reported results on MRPC as part of its GLUE score, and subsequent models like [large-language-model](https://www.wikiprompt.org/wiki/large-language-model)s from [openai](https://www.wikiprompt.org/wiki/openai), [anthropic](https://www.wikiprompt.org/wiki/anthropic), and [google-deepmind](https://www.wikiprompt.org/wiki/google-deepmind) have been evaluated on GLUE tasks, including MRPC, to demonstrate their sentence understanding capabilities.

The task is framed as a binary classification problem: given a pair of sentences, the model must output a label indicating whether they are paraphrases. Performance is typically measured using accuracy and F1 score, with the F1 score being the primary metric in the GLUE leaderboard. State-of-the-art models have achieved F1 scores above 90%, while human performance is estimated around 85-90%, indicating that the task is nearly saturated but still useful for regression testing.

## Influence on Paraphrase Research

Beyond its use as a benchmark, MRPC has influenced research on paraphrase generation and detection. The dataset provides a naturalistic set of paraphrase pairs that have been used to train sequence-to-sequence models for generating paraphrases, as well as to evaluate unsupervised methods. It has also been used in studies of [artificial-intelligence](https://www.wikiprompt.org/wiki/artificial-intelligence) interpretability, where researchers analyze which linguistic features models rely on when making paraphrase judgments.

One limitation of MRPC is its relatively small size and narrow domain (news text), which can lead to overfitting if used alone. To address this, researchers often combine MRPC with other paraphrase datasets, such as the Quora Question Pairs dataset, or use it as a fine-tuning task within multi-task learning frameworks. The corpus has also been criticized for containing annotation noise, but it remains a de facto standard due to its longevity and ease of use.

## Legacy and Continued Use

As of the mid-2020s, MRPC continues to be referenced in academic papers and is included in popular NLP libraries such as Hugging Face's Datasets. It is often used as a sanity check for new architectures, alongside other GLUE tasks like [SST-2](https://www.wikiprompt.org/wiki/sst-2) and [QQP](https://www.wikiprompt.org/wiki/qqp). While newer benchmarks like [SuperGLUE](https://www.wikiprompt.org/wiki/superglue) and [MMLU](https://www.wikiprompt.org/wiki/mmlu) have emerged, MRPC's simplicity and clear task definition make it a useful tool for debugging and for teaching paraphrase detection. The dataset is freely available for research purposes, and its creation by microsoft-research has been a model for subsequent corpus-building efforts in the field.

## See Also

- [GLUE](https://www.wikiprompt.org/wiki/glue)
- [BERT](https://www.wikiprompt.org/wiki/bert)
- Paraphrase
- [Natural language processing](https://www.wikiprompt.org/wiki/natural-language-processing)

---
Source: https://www.wikiprompt.org/wiki/mrpc
License: CC BY-SA 4.0 (https://creativecommons.org/licenses/by-sa/4.0/)
Last updated: 2026-09-07T21:34:44.389298+00:00