The Face Recognition Vendor Test (FRVT) is a series of large-scale, independent evaluations of face recognition systems, organized by the U.S. National Institute of Standards and Technology (NIST). Initiated in 2000, the program has evolved from periodic assessments into an ongoing effort that continuously benchmarks algorithms submitted by vendors and research institutions worldwide. The FRVT measures the accuracy, speed, and robustness of face recognition technology across a variety of image types and operational scenarios, and its results are widely used by government agencies, industry, and researchers to gauge the state of the art.
The FRVT series builds on earlier NIST evaluations, notably the Face Recognition Technology (FERET) evaluations conducted in 1994, 1995, and 1996. While FERET established foundational testing methodologies, the FRVT expanded the scope to include larger datasets, more challenging imagery, and a broader range of tasks. Over time, the program has grown to include specialized sub-tests for face-in-video recognition, detection of facial morphing, and analysis of demographic effects such as age, gender, and race on algorithm performance.
FRVT 2006
The FRVT 2006 evaluation, conducted in 2006, was a major milestone in the series. Its primary goal was to measure progress in prototype and commercial face recognition systems since the FRVT 2002 evaluation. The evaluation assessed performance on high-resolution still imagery (5 to 6 megapixels), 3D facial scans, multi-sample still facial imagery, and pre-processing algorithms designed to compensate for pose and illumination variations.
To ensure an accurate assessment, FRVT 2006 used sequestered data - images not previously seen by researchers or developers - and a standardized test methodology. All participants were evaluated evenly using a common dataset and test environment called the Biometric Experimentation Environment (BEE). The BEE simplified test data management, experiment configuration, and result processing, allowing experimenters to focus on algorithm development. The evaluation was sponsored by multiple U.S. government agencies and managed by NIST.
One objective of FRVT 2006 was to independently determine whether the goals of the Face Recognition Grand Challenge (FRGC) had been achieved. The FRGC, a separate algorithm development project conducted from May 2004 through March 2006, aimed to produce face recognition algorithms an order of magnitude better than those evaluated in FRVT 2002. FRGC data remains available to researchers who sign required licenses and follow data release rules.
FRVT 2006 Results
The large-scale results of FRVT 2006 were published in a combined report with the Iris Challenge Evaluation (ICE) 2006. The evaluation drew input from 22 organizations across 10 countries, with many submitting multiple algorithms. The report documented a dramatic improvement in error rates over time. For example, the false rejection rate (FRR) at a false acceptance rate (FAR) of 0.001 improved from 0.79 in 1993 to 0.01 in 2006 - a reduction of nearly two orders of magnitude. Part of this improvement was attributed to higher-quality face images. The best results in 2006 came from algorithms using very high-resolution still images (6 megapixels) and 3D scans.
Face Recognition Prize Challenge 2017
The Face Recognition Prize Challenge (FRPC), held in 2017, assessed algorithms on photographs collected without tight quality constraints, such as images of individuals who were not cooperating or unaware they were being photographed. Prizes were awarded for both verification and identification tasks. The best verification algorithm achieved a false non-match rate (FNMR) of 0.22 at a false match rate (FMR) of 0.001. Additional prizes were given for processing speed and for verification against a set of cooperative portrait photos.
FRVT Ongoing
The FRVT program is now in an ongoing status with periodic reports. As of recent reports, it evaluates roughly 200 face recognition algorithms against at least six collections of photographs containing multiple images of more than 8 million people. On high-quality visa images, the best algorithms for 1:1 verification achieve false non-match rates of 0.0003 at false match rates of 0.0001.
The ongoing program includes several specialized sub-tests:
- FRVT: Demographic Effects - examines how demographic differences such as age, gender, and race affect algorithm performance.
- FRVT MORPH - focuses on detection of facial morphing, particularly as it relates to photo-credential issuance.
- FACE Challenges - tests recognition of individuals from photographs posted on social media.
- Face in Video Evaluation (FIVE) - assesses the ability of algorithms to identify or ignore persons in video sources, often where the subject is not actively cooperating, i.e., "in the wild."
Sponsors and Impact
The FRVT has been sponsored by several U.S. government agencies, including the Intelligence Advanced Research Projects Agency (IARPA), the Department of Homeland Security (DHS), the FBI Criminal Justice Information Services Division, the Technical Support Working Group (TSWG), and the National Institute of Justice. The program's results have influenced procurement decisions, algorithm development, and public policy discussions around facial recognition technology. By providing independent, standardized evaluations, the FRVT serves as a benchmark for measuring progress in Artificial intelligence and Machine learning applications in biometric identification.
The FRVT's ongoing nature reflects the rapid evolution of face recognition technology, driven by advances in Deep learning and Neural network architectures. As algorithms improve, the evaluation continues to expand its scope, addressing new challenges such as demographic bias and video-based recognition, ensuring that the technology is assessed under realistic and demanding conditions.