Ground truth - definition
Ground truth is a set of verified reference labels used to measure how accurately a system detects objects in images or video. In face and license plate anonymization, ground truth identifies the faces and license plates that should be detected and masked in each image or video frame.
Ground-truth labels are usually created by trained human annotators. They may take the form of bounding boxes, polygons, pixel-level masks, object classes, frame numbers, and visibility attributes. The labels provide the expected result against which a face detection or license plate detection system is evaluated. They are called “ground truth” by convention, but they are not automatically error-free. Their quality depends on annotation rules, reviewer training, image quality, and quality-control procedures.
Role of ground truth in image and video anonymization
Ground truth is required to evaluate whether an artificial intelligence (AI) model detects the faces and license plates that require blurring. It separates a measurable detection result from a subjective visual assessment. Without reference labels, an organization cannot reliably calculate how often the system misses an identifiable face or license plate.
For anonymization workflows, the ground-truth dataset should reflect the footage that will be processed. This can include CCTV recordings, vehicle-mounted cameras, mobile devices, public-event footage, body-worn cameras, or archived video. Relevant variation includes camera angle, distance, motion blur, shadows, weather, compression artifacts, partial occlusion, and night scenes.
Each labeled object normally includes the following information:
- Object category, such as face or license plate.
- Location, commonly represented by a rectangular bounding box.
- Frame identifier or timestamp for video footage.
- Visibility status, such as fully visible, partly occluded, blurred, distant, or truncated at an image boundary.
- Optional review attributes, such as annotation confidence or disagreement between annotators.
How ground truth is created
A ground-truth project begins with a written annotation specification. The specification should define what counts as a face or license plate, how to label partially visible objects, whether reflections are included, and how tightly a box or mask must follow the object boundary. Consistent instructions are necessary because small annotation differences can change evaluation results.
For video, annotators can label every frame or label selected frames and interpolate object positions between them. Interpolation can reduce manual effort, but it must be reviewed where objects move quickly, become occluded, or leave the camera view. A missed face in one frame can create a privacy risk even if the same person is correctly detected in nearby frames.
The table summarizes common ground-truth formats for anonymization testing.
Label format | Content | Typical use
|
|---|---|---|
Bounding box | Rectangle around a face or license plate | Testing object detection and mask placement |
Polygon or segmentation mask | Object boundary at pixel level | Testing precise masking coverage |
Video track | Linked object labels across sequential frames | Testing detection continuity and missed frames |
Ignore region | Area excluded under documented rules | Handling ambiguous, unreadable, or severely occluded objects |
Key metrics for ground-truth evaluation
Evaluation compares predicted detections with ground-truth labels. A predicted face or license plate is generally counted as a true positive only when it has the correct object class and overlaps sufficiently with a reference annotation. The overlap measure is Intersection over Union (IoU).
The IoU formula is:
IoU = area of overlap between prediction and ground truth / area of union of prediction and ground truth
The following metrics should be reported together because no single metric describes anonymization risk completely.
- Recall: the proportion of ground-truth objects detected by the system. Low recall means that faces or license plates may remain unblurred.
- Precision: the proportion of system detections that correspond to ground-truth objects. Low precision can cause unnecessary blurring of non-sensitive image areas.
- False negatives: ground-truth faces or license plates that the system did not detect.
- False positives: detections without a corresponding ground-truth object.
- Average Precision (AP): a precision-recall summary calculated at defined IoU thresholds.
The PASCAL Visual Object Classes (VOC) challenge used an IoU threshold of 0.5 for object detection evaluation. The Microsoft Common Objects in Context (COCO) evaluation protocol reports AP across IoU thresholds from 0.50 to 0.95 in increments of 0.05. These protocols are useful references, but a privacy-focused test plan should also report the absolute number and type of missed faces and license plates, especially in difficult footage.
Ground truth and mask quality
Detection accuracy alone does not prove that anonymization is effective. A system can correctly detect a face while applying a mask that is too small, shifted, intermittent, or insufficiently opaque. Ground truth can therefore be used at two stages: first to assess detection, and then to assess whether the applied blur or mask covers the annotated object.
For example, a review may measure whether the final mask overlaps the face or license plate ground-truth region by a predefined minimum amount. The required threshold should be documented in the test protocol because it depends on the masking method, object size, and privacy objective. Test results should distinguish detection errors from rendering or tracking errors.
Practical use in Gallio PRO workflows
Organizations can use a representative ground-truth sample to validate image and video anonymization before wider processing. This is particularly important when footage comes from a new camera type, location, or recording condition. The sample should be kept separate from data used to tune detection settings so that the reported result reflects performance on unseen material.
Gallio PRO automatically detects and blurs faces and license plates in supported images and video files. It does perform video-stream anonymization in supported workflows. Ground-truth review can identify objects that require manual correction, including missed faces or license plates. Elements not automatically detected, such as company logos, tattoos, name badges, documents, and monitor content, can be blurred manually with the built-in editor when required by the processing purpose.
Standards and references
There is no single universal ground-truth standard for face and license plate anonymization. Evaluation protocols should state the annotation rules, dataset composition, object classes, IoU thresholds, review method, and treatment of ambiguous objects. The following technical references provide established evaluation concepts.
- PASCAL VOC Challenge, 2010 - object detection evaluation using an IoU threshold of 0.5.
- Microsoft COCO: Common Objects in Context, 2014 - AP evaluation across multiple IoU thresholds.
- ISO/IEC 5259-1:2024 - Data quality for analytics and machine learning, including a framework for managing data quality.
- National Institute of Standards and Technology Face Recognition Vendor Test - methodology and reports for biometric face technology evaluation.