Occlusion Handling for Obstructed Faces - Definition
Occlusion handling is a set of methods used to detect, track, and correctly blur faces that are partially obscured in a photo or video. In the context of image and video anonymization, it refers to situations where the system must recognize the presence of a face even when facial features are not fully visible, for example when the face is covered by a mask, glasses, a hand, a helmet, hair, a pole, a piece of clothing, or another person.
From a technical perspective, this is not a separate technology but rather a capability and a set of procedures within a face detection model and the downstream anonymization stage. The goal is not to identify the person, but to maintain a high level of face detectability under conditions of partial visual information loss. In practice, this means the AI model must learn to recognize a face based on incomplete geometric and texture patterns, and then pass its location to the blurring or masking module.
In photo and video anonymization systems, occlusion handling has a direct impact on compliance with the privacy by design principle under Article 25 of the GDPR, because failing to detect a partially obscured face may result in the unintended disclosure of personal data. In deep learning-based models, the neural network must first be trained on datasets that include cases of partial occlusion. Simply using AI without a properly prepared model does not solve the problem.
The Role of Occlusion Handling in Photo and Video Anonymization
Under standard conditions, face detection relies on a complete arrangement of features such as the eyes, nose, mouth, and facial contour. When some of these elements are missing, the risk of a false negative increases - meaning the system fails to detect a face that is actually present in the frame. For the anonymization process, this is a critical error because it leaves a face unblurred.
Occlusion handling is especially important in surveillance footage, dashcam recordings, field documentation, public event footage, and recordings captured in motion. Such data often includes:
- partial face profiles,
- brief obstructions between frames,
- low object resolution,
- strong lighting changes,
- multiple people overlapping in a single scene.
In practice, effective anonymization requires the system not to evaluate each frame in isolation. In video, this is why systems use not only detection but also object tracking across frames, which helps maintain the anonymization mask even when a face becomes partially hidden for several frames.
Technologies Used in Occlusion Handling
Effective occlusion handling for obstructed faces usually relies on a combination of several processing layers. In modern solutions, the foundation is deep learning models, most commonly convolutional neural networks or single-stage and two-stage detectors.
The following technical approaches are used most often:
- face detection trained on datasets containing partial occlusion,
- facial landmark estimation even when some landmarks are not visible,
- object tracking in video sequences using tracking-by-detection methods,
- aggregation of results across frames,
- confidence thresholding with an additional safety rule for borderline areas.
Research literature often uses the WIDER FACE benchmark, published in 2016, which includes significant variation in face scale, pose, expression, and occlusion. This dataset is widely used to evaluate face detectors under challenging scene conditions. Results reported for this benchmark are usually expressed as Average Precision (AP), separately for the Easy, Medium, and Hard levels. The Hard category includes, among other cases, small faces and severe occlusions.
In a system such as Gallio PRO, occlusion handling should be understood as the ability of the automatic face detection module to remain effective even when a face is partially obscured. If automatic detection is not sufficient for a specific file, the correction can be made manually in the editor. However, this applies only to objects supported by the system or manually marked by the user. Gallio PRO automatically blurs faces and license plates. It does not automatically detect logos, tattoos, ID badges, documents, or content displayed on monitor screens.
Key Parameters and Metrics for Occlusion Handling
Evaluating the quality of occlusion handling should not be limited to overall detection performance. Metrics need to be measured on data containing partial and severe occlusions. Otherwise, the result may be misleading from an anonymization perspective.
Parameter / Metric | Meaning | Importance for Anonymization
|
|---|---|---|
Recall | The proportion of actual faces detected by the model | The most important indicator for reducing missed detections |
Precision | The proportion of correct detections among all detections | Affects the number of unnecessary masks |
AP on an occlusion dataset | Average Precision calculated from the precision-recall curve for a given matching criterion, usually at a defined IoU threshold | Enables model comparison under difficult conditions |
IoU - Intersection over Union | The overlap between the detection box and the reference area | Affects whether the blur covers the entire face |
Latency | The time required to process a frame or file | Important for batch performance and near-real-time systems |
False Negative Rate | The proportion of undetected faces | The highest-risk error in legal and operational terms |
In simplified form, recall can be expressed as:
Recall = TP / (TP + FN)
where TP means correctly detected faces and FN means missed faces. In privacy protection applications, a configuration that increases recall is usually preferred, even at the cost of a moderate rise in false positives, because excessive blurring is typically less risky than a failure to anonymize.
Challenges and Limitations of Occlusion Handling
Even strong models have limitations. The problem becomes more difficult when occlusion is combined with a small face size, motion blur, low contrast, and an unusual camera angle. Under these conditions, the system may incorrectly conclude that the object does not contain enough evidence to be classified as a face.
The most common limitations include:
- severe occlusion exceeding the model's information threshold,
- unstable detection between adjacent frames,
- quality degradation for very small faces,
- dependence on training data quality,
- differences between benchmark results and real-world source material.
For this reason, the anonymization process should include qualitative validation, proper confidence threshold selection, and operator review for high-risk materials. This is especially important when the recording is intended for publication or disclosure to a third party.
Normative References and Practical Compliance Context
Occlusion handling is not separately defined in the GDPR or in ISO standards as an independent technical obligation. However, its importance follows from the general principles of personal data processing and processing security. In particular, the following are relevant:
- Regulation (EU) 2016/679 of the European Parliament and of the Council - GDPR, in particular Articles 5, 25, and 32,
- ISO/IEC 23894:2023 - Artificial intelligence - Guidance on risk management,
- ISO/IEC 27001:2022 - Information security management systems,
- NIST AI RMF 1.0, 2023 - guidance for AI system risk management.
From the perspective of a Data Protection Officer, the practical assessment should include not only whether the system detects faces, but also how it performs under occlusion, what level of recall it achieves on difficult data, and whether a manual correction procedure is available. In the case of Gallio PRO, it is also important that the solution operates as on-premise software, does not anonymize video streams or perform real-time anonymization, and does not collect logs containing face and license plate detections.