SROIE (Scanned Receipt OCR and Information Extraction) is a benchmark dataset designed for evaluating optical character recognition (OCR) and information extraction systems on scanned receipts. Introduced in 2019 as part of the ICDAR (International Conference on Document Analysis and Recognition) competition, it consists of 1,000 real-world receipt images, split into 600 training, 100 validation, and 300 test samples. The dataset focuses on extracting key fields such as company name, address, date, and total amount, making it a standard testbed for document understanding models.
The dataset was created by researchers from the Chinese Academy of Sciences and other institutions to address the lack of standardized evaluation for receipt OCR. Unlike synthetic datasets, SROIE uses genuine receipts with varied layouts, fonts, and printing quality, reflecting real-world challenges like noise, skew, and low resolution. Each receipt is annotated with both text-level bounding boxes and structured key-value pairs, enabling tasks like text localization, transcription, and semantic parsing.
Task Structure and Evaluation
SROIE defines three primary tasks: text localization (detecting words and lines), text recognition (transcribing the detected text), and key information extraction (mapping recognized text to predefined fields). The official evaluation metrics include precision, recall, and F1-score for each task, with the extraction task requiring exact field matching. For the extraction task, the system must output a JSON-like structure with fields like "company", "address", "date", and "total". The competition's final ranking is based on the extraction F1-score, which combines accuracy across all fields.
Technical Characteristics
Receipts in SROIE exhibit diverse formats, including thermal prints, dot-matrix prints, and handwritten annotations. The images are typically captured under varying lighting conditions, adding complexity for OCR algorithms. The dataset's annotations are provided in a custom XML format, with each word tagged by its position and text, and each key field linked to the corresponding word or phrase. This structure supports both end-to-end models and modular pipelines that separate detection, recognition, and extraction.
Impact on Document AI
SROIE has become a widely cited benchmark in the field of document understanding and OCR. It has been used to evaluate models based on convolutional neural networks and recurrent neural networks, as well as more recent transformer-based architectures. The dataset has also driven research in data augmentation techniques for receipt images, such as synthetic noise addition and geometric transformations. Many large language models with vision capabilities have been tested on SROIE to assess their ability to perform structured extraction from real-world documents.
Limitations and Extensions
Despite its utility, SROIE has limitations. The dataset is relatively small, with only 1,000 images, which can lead to overfitting in model training. The field set is limited to four key fields, omitting other common receipt elements like taxes or itemized lines. Additionally, the receipts are predominantly from Chinese merchants, which may introduce language and regional biases. Researchers have proposed extensions, such as adding more diverse receipts or creating synthetic variants, but the original SROIE remains the standard for fair comparison.
Related Benchmarks
SROIE is part of a broader family of document understanding datasets, including FUNSD for form understanding and CORD for receipt extraction. While CORD offers more detailed field annotations (e.g., item names, prices, subtotals), SROIE's simplicity and clear evaluation protocol make it a preferred starting point for new models. The dataset has also been incorporated into larger multi-task benchmarks like DocVQA and Scene Text recognition challenges, allowing cross-domain evaluation.
Future Directions
As of 2025, SROIE continues to be used in academic research and industry applications, particularly for automating expense management and accounting workflows. With the rise of generative AI and multimodal models, researchers are exploring how to leverage pre-trained vision-language models to achieve higher extraction accuracy on SROIE without task-specific training. However, the dataset's fixed size and limited field diversity mean that new benchmarks are emerging to address more complex document types, while SROIE remains a historical reference point for OCR and information extraction progress.