CORD (Consolidated Receipt Dataset) is a publicly available dataset designed for research in receipt understanding, a subfield of document understanding within Artificial intelligence. It consists of over 11,000 annotated receipt images collected from real-world sources, covering diverse merchants, layouts, and languages. The dataset provides pixel-level and token-level annotations that support tasks such as key information extraction, semantic segmentation, and entity recognition, making it a benchmark for evaluating models that process receipts.
CORD was introduced in 2019 by researchers from the Naver Corporation and the University of Science and Technology in South Korea. The dataset was created to address the lack of standardized, large-scale annotated data for receipt parsing, which is critical for applications like expense tracking, accounting automation, and mobile payment systems. Since its release, CORD has been widely adopted in academic research and industry, serving as a standard testbed for comparing Deep learning and Machine learning approaches to document intelligence.
The annotations in CORD follow a hierarchical structure that organizes receipt information into categories such as store name, date, time, items, prices, and payment methods. Each text region is labeled with a semantic class, and relationships between regions (e.g., item to price) are captured. This fine-grained annotation enables models to learn not only what information is present but also how it is structured, which is essential for downstream tasks like automated bookkeeping.
Annotation Structure
CORD's annotation schema defines 30 classes across 4 main categories: menu, submenu, item, and option, along with additional fields like quantity, unit price, and total. Each text segment is assigned a class, and spatial relationships between segments are encoded as directed edges. For instance, an item name is linked to its corresponding unit price and quantity. This structure supports both flat classification and relational learning, allowing models to perform joint extraction and relation prediction.
The dataset includes bounding boxes for each text region, as well as the text content and its class label. Annotations are provided in JSON format, facilitating easy integration with modern Neural network pipelines. The hierarchical design also enables evaluation of both token-level and segment-level performance, giving researchers granular metrics.
Applications in AI Research
CORD has become a reference point for developing and benchmarking models in document understanding. It is commonly used to train and test Transformer (architecture)-based architectures, such as those that combine visual and textual features. For example, models like LayoutLM and Donut have been evaluated on CORD, demonstrating its utility in advancing state-of-the-art performance. The dataset also supports research in Generative AI, where models generate structured outputs from raw receipt images.
Beyond academic research, CORD has practical applications in industries that rely on receipt digitization. Companies in fintech, retail, and logistics use models trained on CORD to automate data entry, reduce manual errors, and improve customer experiences. The dataset's diversity in receipt formats helps ensure that models generalize to real-world variations, such as different fonts, orientations, and lighting conditions.
Technical Challenges
Receipt understanding poses several challenges that CORD helps researchers address. Receipts often contain noisy text, overlapping elements, and irregular layouts, making accurate segmentation difficult. Additionally, the variety of languages and currencies in CORD requires models to handle multilingual and multi-format inputs. Another challenge is the small size of some text regions, which can be missed by detection systems. CORD's detailed annotations allow researchers to isolate these issues and develop targeted solutions, such as improved Data Augmentation techniques or Multi-Head Attention mechanisms.
Related Datasets and Benchmarks
CORD is often compared with other receipt and document datasets, such as SROIE (Scanned Receipts OCR and Information Extraction) and FUNSD (Form Understanding in Noisy Documents). While SROIE focuses on key information extraction from receipts, CORD provides richer annotations, including semantic segmentation and relation extraction. This makes CORD more suitable for tasks that require a deeper understanding of document structure. Researchers frequently use CORD alongside these datasets to evaluate the generalization of their models across different document types.
Future Directions
The field of document understanding is evolving rapidly, with Large language models being adapted for multimodal tasks. CORD is likely to remain a valuable resource for training and evaluating such models, especially as they become more capable of handling complex layouts and languages. Future extensions of CORD could include more diverse receipt formats, additional languages, and annotations for new tasks like table extraction or anomaly detection. As Artificial intelligence continues to advance, datasets like CORD will play a crucial role in ensuring that models are robust, accurate, and applicable to real-world scenarios.