# R-CNN

R-CNN is a family of deep learning models for object detection and localization, using region proposals and convolutional neural networks to identify objects in images.

Region-based Convolutional Neural Networks (R-CNN) are a family of machine learning models for computer vision, specifically object detection and localization. The original goal of R-CNN was to take an input image and produce a set of bounding boxes as output, where each bounding box contains an object and also the category (e.g. car or pedestrian) of the object. In general, R-CNN architectures perform selective search over feature maps outputted by a CNN.

R-CNN has been extended to perform other computer vision tasks, such as tracking objects from a drone-mounted camera, locating text in an image, and enabling object detection in Google Lens. Mask R-CNN is also one of seven tasks in the MLPerf Training Benchmark, a competition to speed up the training of neural networks.

## History

The development of R-CNN has seen several key versions. The original R-CNN was introduced in November 2013, followed by Fast R-CNN in April 2015 and Faster R-CNN in June 2015. Mask R-CNN arrived in March 2017, extending the framework to instance segmentation. In December 2017, Cascade R-CNN was proposed, which trains with increasing Intersection over Union (IoU, also known as the Jaccard index) thresholds, making each stage more selective against nearby false positives. In June 2019, Mesh R-CNN added the ability to generate a 3D mesh from a 2D image.

## Architecture

The R-CNN family shares a common architecture based on region proposals and convolutional feature extraction. The core idea is to generate candidate object regions, extract features using a CNN, and classify each region.

### Selective Search

Given an image (or an image-like feature map), selective search (also called Hierarchical Grouping) first segments the image using the algorithm of Felzenszwalb and Huttenlocher (2004). It then iteratively merges similar neighboring regions based on color, texture, size, and shape compatibility, producing a set of object location hypotheses. The algorithm proceeds as follows:

1. Segment the image into initial regions R = {r1, ..., rn}.
2. Initialize a similarity set S = ∅.
3. For each neighboring region pair (ri, rj), calculate similarity s(ri, rj) and add to S.
4. While S is not empty:
  - Get the highest similarity s(ri, rj) = max(S).
  - Merge the corresponding regions rt = ri ∪ rj.
  - Remove similarities involving ri and rj from S.
  - Calculate similarities between rt and its neighbors, add to S.
  - Add rt to R.
5. Extract object location boxes L from all regions in R.

### R-CNN

With the original R-CNN, prediction follows a two-step process. A preprocessing selective search step generates a large set of candidate objects (typically as many as 2000), known as regions of interest (ROI). These are forwarded to a CNN, which predicts an object class score and bounding box estimate independently for each ROI. Importantly, the ROIs are heavily filtered to remove excess candidates. Filtering begins by removing ROIs assigned to the background category, a specialized category scored by the CNN alongside other categories. Remaining ROIs often suffer from heavy duplication, as multiple ROIs covering the same object are all assigned non-background categories. This is resolved by a heuristic non-maximum suppression (NMS) step.

### Fast R-CNN

While the original R-CNN independently computed neural network features on each of up to two thousand regions of interest, Fast R-CNN runs the neural network once on the whole image. At the end of the network is a ROIPooling module, which slices out each ROI from the network's output tensor, reshapes it, and classifies it. As in the original R-CNN, Fast R-CNN uses selective search to generate its region proposals.

### Faster R-CNN

Faster R-CNN integrates the ROI generation into the neural network itself, eliminating the need for external selective search. This makes the model fully trainable end-to-end and significantly faster.

### Mask R-CNN

While previous versions focused on object detection, Mask R-CNN adds instance segmentation, producing a segmentation mask for each detected object. Mask R-CNN also replaced ROIPooling with a new method called ROIAlign, which can represent fractions of a pixel, improving localization accuracy.

## Applications and Impact

R-CNN models have been widely adopted in computer vision applications, including autonomous driving, surveillance, and image search. The architecture has influenced subsequent object detection models and remains a foundational concept in deep learning for vision. The MLPerf benchmark includes Mask R-CNN as a standard task for evaluating training performance, highlighting its practical importance.

## See Also

- [deep-learning](https://www.wikiprompt.org/wiki/deep-learning)
- [neural-network](https://www.wikiprompt.org/wiki/neural-network)
- [residual-network](https://www.wikiprompt.org/wiki/residual-network)
- [u-net](https://www.wikiprompt.org/wiki/u-net)

## References

- Girshick, R., Donahue, J., Darrell, T., & Malik, J. (2014). Rich feature hierarchies for accurate object detection and semantic segmentation. CVPR.
- Girshick, R. (2015). Fast R-CNN. ICCV.
- Ren, S., He, K., Girshick, R., & Sun, J. (2015). Faster R-CNN: Towards real-time object detection with region proposal networks. NeurIPS.
- He, K., Gkioxari, G., Dollár, P., & Girshick, R. (2017). Mask R-CNN. ICCV.

## Further Reading

- Parthasarathy, Dhruv (2017-04-27). "A Brief History of CNNs in Image Segmentation: From R-CNN to Mask R-CNN". Medium. Retrieved 2024-09-11.

---
Source: https://www.wikiprompt.org/wiki/r-cnn
License: CC BY-SA 4.0 (https://creativecommons.org/licenses/by-sa/4.0/)
Last updated: 2026-09-13T04:00:05.720901+00:00