Intersection over Union (IoU) - definition
Intersection over Union (IoU) is a metric that measures how closely a predicted detection area matches the annotated area of a real object. It is commonly used to evaluate object detection systems, including systems that detect faces and license plates in images or video frames before those areas are blurred.
IoU compares two regions: a predicted bounding box generated by a detection model and a ground truth bounding box created by an annotator. The result ranges from 0 to 1. An IoU value of 1 means that both regions are identical. An IoU value of 0 means that they do not overlap.
The metric is calculated as follows:
IoU = Area of overlap / Area of union
The area of union includes all pixels covered by either the predicted bounding box or the ground truth bounding box. The overlap includes only pixels covered by both boxes.
IoU value | Interpretation
|
|---|---|
0.00 | No overlap between the predicted box and the ground truth box. |
0.50 | Partial overlap. This is a commonly used minimum threshold for a correct detection in benchmark evaluation. |
0.75 | Strong alignment between the predicted box and the annotated object. |
1.00 | Perfect alignment between the predicted and annotated boxes. |
IoU is a technical evaluation metric. It does not itself establish whether image or video processing is lawful, whether footage is anonymous, or whether a blur effect adequately protects a person’s identity. It helps assess whether the detection stage identifies the correct image region for subsequent anonymization.
How Intersection over Union applies to face and license plate blurring
Automated image anonymization generally requires several separate steps. A model first performs face detection or license plate detection. It then outputs a bounding box and a confidence score. The system can apply a blur, pixelation, masking, or other visual transformation to the detected area.
IoU is particularly relevant during model testing because a detection box that is too small may leave identifiable parts of a face or license plate visible. A box that is too large may obscure unnecessary image content. For privacy-oriented processing, evaluators should examine both the IoU score and the visual coverage produced by the final anonymization effect.
- Under-sized detection box: The bounding box may not cover the full face, hairline, or license plate boundaries. The resulting blur can leave identifying details visible.
- Over-sized detection box: The box may cover nearby people, road signs, vehicle details, or other contextual content that is not necessary to obscure.
- Correct box but insufficient blur: A detection can have high IoU while the applied blur remains too weak to prevent identification.
- Missed object: IoU cannot be calculated for an object that the system does not detect. Recall is needed to measure missed faces or license plates.
For this reason, IoU should not be used as the only acceptance criterion for an image or video anonymization workflow. It should be evaluated with recall, precision, false negative rate, confidence thresholds, and manual quality checks.
IoU thresholds in detection evaluation
An IoU threshold determines when a predicted bounding box is counted as matching a ground truth object. If the measured IoU meets or exceeds the selected threshold, the prediction can be classified as a true positive. If it falls below the threshold, it is usually treated as a false positive, even if it partially overlaps the object.
The appropriate threshold depends on the use case, object size, annotation rules, and privacy risk. Small faces in distant video frames and angled license plates can be difficult to annotate and detect precisely. A single fixed threshold may therefore hide important weaknesses.
Evaluation approach | IoU threshold method | Practical meaning
|
|---|---|---|
PASCAL Visual Object Classes (VOC) | IoU of 0.50 | A detection is generally accepted when its overlap with the ground truth is at least 50 percent. |
Common Objects in Context (COCO) | Average over IoU thresholds from 0.50 to 0.95 in increments of 0.05 | The metric measures performance at both moderate and strict localization requirements. |
Privacy-focused internal testing | Defined by the organization | The threshold should be documented together with blur expansion settings, review rules, and risk tolerance. |
The PASCAL VOC challenge used an IoU threshold of 0.5 for object detection evaluation. The COCO benchmark uses average precision across ten thresholds, starting at 0.50 and ending at 0.95. COCO-style evaluation is stricter because a detection that passes at 0.50 may fail at higher thresholds.
IoU, precision, and recall in anonymization quality testing
IoU measures localization quality for matched detections. It does not measure all detection errors. A robust evaluation of face blurring or license plate blurring should combine IoU with metrics that address whether objects were found and whether incorrect objects were selected.
- Precision: The proportion of detected boxes that correspond to actual faces or license plates. Low precision can cause unnecessary blurring.
- Recall: The proportion of actual faces or license plates that were detected. Low recall creates a risk that identifiable objects remain visible.
- False negatives: Faces or license plates present in the footage but not detected by the model.
- False positives: Areas incorrectly detected as faces or license plates.
- Average Precision (AP): A summary measure that combines precision and recall at a specified IoU threshold or across multiple thresholds.
For video footage, evaluation should also account for frame-to-frame behavior. A detector may achieve acceptable IoU on individual frames but intermittently miss a face during movement, occlusion, motion blur, poor lighting, or camera angle changes. Testing should therefore include representative sequences rather than isolated still images only.
Limitations of Intersection over Union for privacy protection
IoU is useful for bounding box evaluation, but it has limitations in privacy-sensitive image processing. It treats all pixels within a box equally. It does not determine whether the box contains identifying features, whether the person can still be recognized from context, or whether a transformed image meets an organization’s privacy requirements.
IoU can also be sensitive to object size. A small shift in a bounding box can substantially reduce IoU for a distant face or a small license plate. Conversely, a relatively high IoU may still be insufficient if the predicted box excludes a critical identifying portion of the object.
Annotation quality is another limiting factor. Ground truth boxes must follow documented annotation instructions. Evaluators should define whether boxes include hair, ears, face coverings, reflections, partially visible plates, and motion-blurred objects. Different annotation policies can produce different IoU results for the same detection model.
Using IoU when evaluating image and video anonymization tools
Organizations evaluating a face and license plate anonymization tool should request test results that identify the dataset, annotation method, object categories, IoU thresholds, confidence thresholds, and error definitions. Results without this context are difficult to compare.
A practical evaluation process can include the following steps:
- Create or obtain a representative test dataset of images or video frames with manually annotated faces and license plates.
- Run the detection process using documented configuration settings.
- Match predicted boxes to ground truth boxes using a defined IoU threshold.
- Calculate precision, recall, false positives, false negatives, and average precision.
- Review samples visually after blurring, especially detections near the selected IoU threshold.
- Test difficult conditions such as low resolution, shadows, partial occlusion, side profiles, moving vehicles, and crowded scenes.
Gallio PRO uses face detection and license plate detection to support image and video anonymization workflows. Its automatic processing covers faces and license plates. Other elements, such as logos, tattoos, name badges, documents, and monitor content, require manual review and manual editing when they must be obscured.
Standards and references
IoU is widely used in computer vision research and benchmark evaluation, but it is not defined by a single universal technical standard. The following primary publications are widely cited sources for its use in object detection evaluation.
- PASCAL Visual Object Classes Challenge, International Journal of Computer Vision, 2010 - defines the overlap criterion used in the PASCAL VOC detection challenge.
- Common Objects in Context (COCO) Detection Evaluation - documents average precision evaluation across IoU thresholds from 0.50 through 0.95.
- Microsoft COCO: Common Objects in Context, 2014 - introduces the COCO dataset and its object detection evaluation framework.
- National Institute of Standards and Technology (NIST), Face Recognition Vendor Test (FRVT) - provides a separate example of structured biometric performance testing, although FRVT metrics are not a substitute for bounding-box IoU evaluation.