IRIS Matching
Goal
- Evaluation Strategies that help me find which are the metrics that I should focus
- To find algorithms that can perform 1-1 and 1-N IRIS
- Evaluation against right data set
Metrics (summary)
- Key Metrics to measure: Accuracy, FNMR, FPMR, matching speed, ROC curve
1-1 IRIS matching (summary)
….
1-N IRIS matching (summary)
…
Things to be concerned about
Probabilistic Nature of Biometric Systems
It seems intuitively obvious that a declared nonmatch in a biometric system with both FMRs and FNMRs of 0.1 percent is almost certainly correct. Unfortunately, intuition is grossly misleading in this instance, and the common misconception can have profound sociological impacts (for example, it might lead to the assumption that a suspected criminal is guilty if the fingerprints or DNA samples from the suspect “match” those at the crime scene). Understanding why this natural belief is often wrong is one of the keys to understanding how to use biometrics effectively. From the perspective of statistical decision theory, it is not enough to focus on error rates. All they provide is the conditional probability of a recognition/nonrecognition given that the presenting individual should be recognized and the conditional probability of a nonrecognition given that the presenting individual should not be recognized.
Metrics dump
FAR and FRR
FAR and FRR refer to results at a broader system level and include failures arising from additional factors, such as the inability to acquire a sample Basically this is done for the whole system rather than one thing
Measures of Operational Efficacy
Key aspects of operational efficacy include recognition error rates; speed; cost of acquisition, operation, and maintenance; data security and privacy; usability; and user acceptance. Generally, trade-offs must be made across all of these measures to achieve the best-performing system consistent with operational and budgetary needs. For example, recognition error rates might be improved by using a better but more time-consuming enrollment process; however, the time added to the enrollment process could result in queues (with loss of user acceptance) and unacceptable costs.
In this report the committee usually discusses recognition error rates in terms of the false match rate (FMR; the probability that the matcher recognizes an individual as a different enrolled subject) and the false nonmatch rate (FNMR; the probability that the matcher does not recognize a previously enrolled subject). FMR and FNMR refer to errors in the matching process and are closely related to the more frequently reported false acceptance rate (FAR) and the false rejection rate (FRR). FAR and FRR refer to results at a broader system level and include failures arising from additional factors, such as the inability to acquire a sample. The committee uses these terms less frequently as they can sometimes intro duce confusion between the semantics of “acceptance” and “rejection” in terms of the claimed performance for biometric recognition versus that for the overall application. For example, in a positive recognition system, a false acceptance occurs when subjects are recognized who should not be recognized—either because they are not enrolled in the system or they are someone other than the subject being claimed. In this case the sense of false acceptance is aligned for both the biometric matching operation and the application function. In a system designed to detect and prevent multiple enrollments of a single person, sometimes referred to as a negative recognition system, a false acceptance results when the system fails to match the submitted biometric sample to a reference already in the database. If the system falsely matches a submitted biometric sample to a reference from a different person, the false match results in a denial of access to system resources (a false rejection).
Look for within-person and between person variability.
This
When within-person variation is small relative to between-person variation, large biometric systems with high accuracy are feasible because the distributions of observed biometric data from different individuals are likely to remain widely separated, even for large groups. When within-person variation is high relative to between-person variation, however, the distributions are more likely to impinge on each other, limiting the capacity of a recognition system. In other words, the number of enrolled subjects cannot be arbitrarily increased for a fixed set of features and matching algorithms without degrading accuracy.
Footnotes:
Empirical Evidence for Correct Iris Match Score Degradation with Increased Time-Lapse Between Gallery and Probe Matches - Sarah E. Baker, Kevin W. Bowyer, and Patrick J. Flynn.