Precision and recall - definition
Precision and recall are performance measures used to evaluate a detector. In image and video anonymization, they show how reliably a system identifies faces or license plates that must be blurred. They are calculated by comparing detector results with ground truth, meaning a reviewed reference set that identifies the relevant faces or license plates in each image or video frame.
Precision measures the proportion of reported detections that are correct. Recall measures the proportion of all relevant objects that the detector successfully finds. For privacy protection, recall is often the more safety-critical measure because a false negative can leave a face or license plate visible in released footage.
Metric | Formula | Meaning for anonymization
|
|---|---|---|
Precision | TP / (TP + FP) | How often a reported face or license plate detection is correct. |
Recall | TP / (TP + FN) | How many faces or license plates present in the footage are detected. |
F1 score | 2 × (precision × recall) / (precision + recall) | A harmonic mean that summarizes precision and recall in one value. |
In these formulas, TP means true positive, FP means false positive, and FN means false negative. A true positive is a correctly detected face or license plate. A false positive is an incorrect detection, such as a detector marking a non-face object as a face. A false negative is a missed face or license plate that should have been detected and blurred.
Why recall matters for face and license plate blurring
Face detection and license plate detection are used before automatic masking or blurring. A detector with insufficient recall can miss a person at the edge of a frame, a small or partly obscured face, or a license plate affected by glare, motion blur, darkness, camera angle, or compression artifacts. If the processing workflow relies only on automatic results, those missed objects can remain identifiable.
For this reason, organizations should assess recall on footage that resembles their actual material. Relevant conditions commonly include:
- small, distant, profile-view, partially covered, or heavily compressed faces;
- license plates captured at oblique angles or under reflections, shadows, rain, and low light;
- rapid camera movement, dropped frames, scene cuts, and object occlusion;
- crowded scenes with overlapping people, vehicles, or repeated objects;
- different camera models, resolutions, frame rates, geographic regions, and plate formats.
A high aggregate recall score does not prove that all operational situations are safe. For example, a detector can achieve strong recall on well-lit frontal faces while performing less reliably on distant faces in nighttime footage. Evaluation should therefore report results by scenario, not only as one overall score.
The precision-recall tradeoff
Many detection models produce a confidence score for each candidate object. A processing workflow accepts the candidate as a detection only when its score meets a selected threshold. Lowering the threshold usually increases recall because more possible faces or plates are accepted. It can also reduce precision because more incorrect objects are accepted.
This tradeoff has different operational effects:
- Low recall: fewer objects are found, which increases the risk that a face or license plate remains unblurred.
- Low precision: more non-sensitive objects may be blurred, which can reduce the usefulness or visual quality of the output.
- High recall with manual review: additional detections can be checked and corrected before footage is released.
There is no universal threshold that is appropriate for every footage type. A threshold should be selected using a documented validation set, the organization’s release criteria, and the available quality-assurance process. The same threshold may not produce the same precision and recall after a change in camera source, video encoding, scene type, or model version.
How precision and recall are measured
Measurement requires labeled test data that was not used to train or tune the detector. For each image or frame, reviewers identify the faces or license plates that should be detected. The evaluation system then matches detections to those reference objects using a defined overlap rule.
For object detection, the common overlap measure is Intersection over Union (IoU). IoU compares the overlap between the predicted bounding box and the ground-truth bounding box with their combined area:
IoU = area of overlap / area of union
A detection is counted as a true positive only when it meets the selected IoU matching threshold and is assigned to the correct reference object. The Common Objects in Context (COCO) detection evaluation protocol reports average precision across IoU thresholds from 0.50 through 0.95 in increments of 0.05. This approach illustrates why a single precision value is incomplete: a detector may identify an object generally while placing a bounding box too inaccurately for reliable masking.
Video-specific evaluation considerations
Video anonymization introduces risks that are not visible in still-image testing. A face may be detected in one frame but missed in adjacent frames. A short gap can expose identifying information, especially when a person moves quickly or appears briefly.
Evaluation should therefore include both frame-level and sequence-level analysis. Frame-level recall measures how many relevant objects are found in individual frames. Sequence-level review checks whether a face or license plate remains continuously covered while it is visible. Useful quality-control measures include missed-object counts, consecutive missed frames, and the duration of any unmasked exposure. These parameters should be defined before testing so that results can be reproduced and compared across model versions.
Application in Gallio PRO workflows
Gallio PRO uses face detection and license plate detection to support automatic blurring in uploaded image and video files. It does not perform real-time anonymization or live video stream anonymization. Precision and recall are relevant when validating whether automatic detection results require additional human review before footage is shared, published, or retained for another authorized purpose.
Automatic detection in Gallio PRO covers faces and license plates only. It does not automatically detect company logos, tattoos, name badges, documents, or content displayed on monitors. These elements can be blurred manually with the built-in editor. Manual review also provides a control for false negatives and false positives in automatically detected faces and license plates.
Standards and references
Precision and recall are established information-retrieval and machine-learning measures. The following primary and technical sources explain their use in classification and object-detection evaluation.
- Davis, J. and Goadrich, M. (2006), “The Relationship Between Precision-Recall and ROC Curves,” Proceedings of the 23rd International Conference on Machine Learning.
- Powers, D. M. W. (2011), “Evaluation: From Precision, Recall and F-Measure to ROC, Informedness, Markedness and Correlation,” Pattern Recognition Letters, 32(15).
- COCO Consortium, Object Detection Evaluation Protocol.
- National Institute of Standards and Technology (NIST), Face Recognition Vendor Test (FRVT). NIST publishes repeatable testing methods and reports for face-related technology evaluation.