Precision and recall are two fundamental performance metrics used to evaluate the quality of a classification or information retrieval system. They quantify how well a model identifies relevant items from a dataset, balancing the costs of false positives and false negatives. Precision, also called positive predictive value, measures the fraction of retrieved instances that are actually relevant. Recall, also known as sensitivity, measures the fraction of all relevant instances that were successfully retrieved. Together, they provide a more nuanced view of a model's behavior than accuracy alone, particularly when classes are imbalanced or when the costs of different errors vary.
In formal terms, precision is calculated as the number of true positives divided by the total number of items the system labeled as positive (true positives plus false positives). Recall is calculated as the number of true positives divided by the total number of items that actually belong to the positive class (true positives plus false negatives). For example, consider a computer program designed to recognize dogs in digital photographs. If a picture contains ten cats and twelve dogs, and the program identifies eight elements as dogs, of which five are actually dogs (true positives) and three are cats (false positives), then seven dogs are missed (false negatives) and seven cats are correctly excluded (true negatives). The program's precision is 5/8, since only five of the eight selected elements are relevant. Its recall is 5/12, since only five of the twelve relevant dogs were found.
Relationship to Statistical Errors
Precision and recall map directly onto concepts from statistical hypothesis testing. If the null hypothesis is that a given item is irrelevant, then a false positive (type I error) occurs when an irrelevant item is incorrectly selected, and a false negative (type II error) occurs when a relevant item is missed. Perfect precision corresponds to zero false positives, meaning every retrieved item is relevant. Perfect recall corresponds to zero false negatives, meaning every relevant item is retrieved. Recall is simply the complement of the type II error rate, calculated as one minus that rate. Precision is related to the type I error rate but in a more complex way, because it also depends on the prior distribution of relevant versus irrelevant items in the population. In the dog recognition example, there are three type I errors out of ten total cats, giving a type I error rate of 3/10, and seven type II errors out of twelve dogs, giving a type II error rate of 7/12.
Interpretation and Utility
Precision is often described as a measure of quality, indicating how trustworthy the positive predictions are. Recall is a measure of quantity, indicating how complete the system's coverage of the positive class is. High precision means the algorithm returns mostly relevant results, while high recall means it returns most of the relevant results, even if some irrelevant ones are also included. These metrics are not particularly useful in isolation. A system can achieve perfect recall by simply retrieving every item in the collection, but such a system would have very low precision. Conversely, a system can achieve perfect precision by selecting only a single item that is almost certainly relevant, but this would sacrifice recall. In practice, precision and recall are usually reported together, often through a precision-recall curve that plots precision as a function of recall. Typically, precision decreases as recall increases, reflecting the inherent tradeoff between the two.
The Precision-Recall Tradeoff
There is often an inverse relationship between precision and recall, where increasing one comes at the cost of reducing the other. The optimal balance depends on the specific application and the relative costs of different types of errors. A smoke detector is a classic example of a system designed with recall in mind. It is intentionally prone to false alarms (type I errors) because the cost of failing to alert during a real fire is prohibitively high. The detector sacrifices precision to ensure it catches all genuine dangers. In contrast, the criminal justice system, guided by Blackstone's ratio that "it is better that ten guilty persons escape than that one innocent suffer," emphasizes precision. The system is designed to avoid convicting innocent people, even if this means letting more guilty individuals go free, thus sacrificing recall.
A brain surgeon removing a cancerous tumor illustrates the tradeoff at an individual level. The surgeon must remove all tumor cells to prevent regrowth but must also avoid removing healthy brain cells to preserve function. A more liberal approach, removing a wider area around the tumor, increases recall by ensuring all cancer cells are extracted, but it also increases the chance of removing healthy tissue. A more conservative approach increases precision by targeting only confirmed cancer cells, but it risks leaving some tumor cells behind. Both outcomes have significant consequences, and the surgeon's choice reflects the relative weights placed on precision and recall in that context.
Use in Machine Learning
In Machine learning, precision and recall are standard evaluation metrics for classification models, particularly in domains with imbalanced datasets. For example, in medical diagnostics, a model detecting a rare disease might have high accuracy simply by predicting the majority class, but precision and recall reveal its actual usefulness. In Artificial intelligence research, these metrics are used to compare models across tasks such as object detection, spam filtering, and document retrieval. They are also foundational to derived metrics like the F1 score, which is the harmonic mean of precision and recall, providing a single number that balances both concerns. The concepts extend beyond binary classification to multi-class settings, where precision and recall are computed per class and then aggregated.
Applications in Information Retrieval
Precision and recall originated in the field of information retrieval, where they evaluate search engines and database query systems. In this context, precision measures the proportion of retrieved documents that are relevant to the user's query, while recall measures the proportion of all relevant documents in the collection that were retrieved. Search engines often face a tradeoff: returning more results can increase recall but may dilute precision with irrelevant pages. Ranking systems use precision at a fixed number of results, such as precision at 10, to evaluate the quality of the top results. These metrics remain central to evaluating modern retrieval systems, including those powered by Large language models and Generative AI technologies.
Limitations and Extensions
Precision and recall have limitations. They do not account for true negatives, which can be important in some applications. They are also sensitive to the choice of the positive class, and their interpretation depends on the prior probability of the positive class. When the positive class is rare, recall can be difficult to improve without introducing many false positives. To address these issues, researchers have developed related metrics such as the F1 score, the precision-recall area under the curve, and the Matthews correlation coefficient. In practice, the choice of metric depends on the application's goals, and precision and recall are often reported alongside other statistics to provide a complete picture of model performance.