Deep Learning - Definition
Deep learning is a subset of machine learning that uses multi-layered artificial neural networks to learn data representations and solve complex perceptual tasks such as object detection, image segmentation, and image classification. This definition is reflected, among others, in ISO/IEC 22989:2022, which standardizes artificial intelligence terminology (source: ISO/IEC 22989:2022).
In the context of image and video anonymization, deep learning forms the foundation for training face detection and license plate detection models. These models enable the automatic blurring or pixelation of sensitive areas while maintaining visual consistency and compliance with data protection regulations.
The Role of Deep Learning in Image and Video Anonymization
To effectively blur faces and license plates, reliable detection models are required for both still images and video sequences. Deep learning provides such models through convolutional and hybrid architectures capable of detecting objects under varying lighting conditions, perspectives, and scales.
In Gallio PRO, deep learning models are used to automatically identify facial regions and license plates, after which a blur or pixelation filter is applied. The software does not automatically detect logos, tattoos, name badges, or screen content—these elements can be manually blurred in the editor. Gallio PRO does not support real-time anonymization or live video stream anonymization. Automated processing is limited strictly to faces and license plates.
Deep Learning Technologies Used for Face and License Plate Blurring
Detection of areas requiring anonymization is based on established model classes and evaluation procedures. Below are the most commonly used approaches, along with scientific references and evaluation standards.
- Face detection: Single-stage convolutional detectors such as RetinaFace and SCRFD combine feature extraction with facial landmark detection, enabling stable masking even when the head is tilted (sources: Deng et al., CVPR 2020 - RetinaFace; Guo et al., 2021 - SCRFD; WIDER FACE dataset - Yang et al., 2016).
- License plate detection: General-purpose object detectors fine-tuned for the “license plate” class and specialized solutions within the ALPR (Automatic License Plate Recognition) pipeline are commonly used. Public datasets such as UFPR-ALPR and AOLP support research and validation (sources: Gonçalves et al., 2018 - UFPR-ALPR; Hsieh et al., 2011 - AOLP).
- Video tracking: After detection in consecutive frames, association algorithms such as DeepSORT or ByteTrack ensure consistent masking of the same object throughout a shot and reduce mask flickering (sources: Wojke et al., 2017 - DeepSORT; Zhang et al., 2022 - ByteTrack).
- Pre- and post-processing: Input normalization, proportional image scaling, Non-Maximum Suppression (NMS) or Soft-NMS to eliminate duplicate detections, followed by deterministic blur or mosaic filters applied to designated regions. Evaluation follows VOC/COCO methodologies using IoU and mAP metrics (sources: Everingham et al., IJCV 2015 - PASCAL VOC; Lin et al., ECCV 2014 - COCO).
Key Parameters and Metrics (Deep Learning for Anonymization)
In practice, Data Protection Officers (DPOs) and technical teams require measurable criteria. The following metrics and attributes help determine whether a deep learning-based anonymization system achieves the expected level of quality and processing security.
Metric / Attribute | Definition | How to Measure | Importance for Anonymization
|
|---|---|---|---|
Precision and Recall | Precision - proportion of detections that are true positives. Recall - proportion of actual objects correctly detected. | Measured on annotated validation datasets. According to VOC/COCO practice, an object is considered detected when IoU exceeds a defined threshold (VOC historically IoU=0.5; COCO averages AP across IoU thresholds from 0.5 to 0.95 in 0.05 increments - sources: VOC, COCO). | In anonymization, high Recall is critical to minimize the risk of leaving a face or license plate unblurred. |
mAP | Mean Average Precision across classes; in COCO metrics, additionally averaged across IoU thresholds. | [email protected] (VOC) or mAP@[0.5:0.95] (COCO) according to standardized evaluation protocols. | Provides a comprehensive assessment of face and license plate detectors across varying overlap levels. |
IoU | Intersection over Union - a measure of overlap between prediction and ground truth. IoU = intersection area / union area. | Calculated for each object. Threshold selection depends on the application and error acceptance policy. | Higher IoU ensures more precise masks and reduces the risk of partial identity exposure. |
Latency and FPS | Processing delay per frame and throughput in frames per second. | End-to-end measurement on the target on-premise infrastructure using representative resolutions. | Affects project turnaround time and compute resource planning, although Gallio PRO does not operate in real time. |
Tracking Stability | Quality of maintaining object identity across frames in multi-object tracking. | Metrics such as IDF1 and HOTA (standard MOT evaluation metrics). | Reduces flickering and masking interruptions in longer video sequences. |
Data Drift | Shift in input data distribution compared to training data. | Ongoing statistical monitoring and periodic validation; according to ISO/IEC 23053:2022, ML lifecycle management should include system supervision. | Prevents Recall degradation on new cameras, nighttime scenes, or different domains. |
Challenges and Limitations
The effectiveness of deep learning in video anonymization depends on data quality, model selection, and risk control. Key considerations for system design and audit include:
- Challenging conditions: Rapid camera motion, motion blur, extreme angles, and occlusions hinder detection. Proper architecture selection and training augmentation are critical (WIDER FACE documents such challenges).
- Domain drift: New camera types, compression artifacts, IR lighting, and varying license plate styles may reduce Recall. Periodic validation and possible model fine-tuning are required (ISO/IEC 23053:2022).
- Privacy risk: False negatives pose GDPR compliance risks. Defensive strategies include lowering confidence thresholds for the “face” class and manual review of borderline cases (privacy by design - GDPR Art. 25).
- Legal considerations: EDPB Guidelines 3/2019 state that a person’s image (and, in certain contexts, identifiers such as license plate numbers) may constitute personal data, justifying anonymization before footage disclosure. Interpretations may vary across jurisdictions; therefore, organizations should adopt a precautionary approach based on the data minimization principle (GDPR Art. 5(1)(c)).
- Functional scope: Gallio PRO automatically blurs only faces and license plates. Other elements can be masked manually in the editor. The software operates on-premise and does not collect logs containing detections, personal data, or sensitive data.
Use Cases and Operational Practice in Gallio PRO
A typical processing workflow includes data preparation, model inference, and validation. The on-premise version ensures full data control and supports compliance with GDPR Article 32 and AI risk management best practices.
- Preparation: Import footage, select an anonymization profile, configure filter type (blur or mosaic), and define operational exceptions if needed.
- Inference: Detect faces and license plates, perform cross-frame association, generate stable masks, and apply visual effects. No live processing. Batch processing is available using local GPU/CPU resources.
- Quality control: Review samples and apply manual corrections if necessary. For high-risk projects, statistical sampling and Recall/Precision reporting on a control dataset are recommended.
- Lifecycle management: Monitor data drift, update models according to ISO/IEC 23053:2022, and manage risk in line with ISO/IEC 23894:2023 and NIST AI RMF 1.0 (2023).
Standards and References
- ISO/IEC 22989:2022 - Artificial intelligence concepts and terminology.
- ISO/IEC 23053:2022 - Framework for AI systems using machine learning.
- ISO/IEC 23894:2023 - Artificial intelligence risk management.
- ISO/IEC 20889:2018 - Privacy-enhancing data de-identification terminology and classification of techniques.
- Regulation (EU) 2016/679 (GDPR) - in particular Articles 4(1), 5(1)(c), 25, and 32.
- EDPB Guidelines 3/2019 on processing of personal data through video devices (adopted 29 January 2020).
- NIST AI Risk Management Framework 1.0, 2023.
- Everingham et al., PASCAL VOC, IJCV 2015.
- Lin et al., Microsoft COCO, ECCV 2014.
- Yang et al., WIDER FACE, CVPR 2016.
- Deng et al., RetinaFace, CVPR 2020.
- Guo et al., SCRFD, 2021.
- Wojke et al., DeepSORT, 2017.
- Zhang et al., ByteTrack, ECCV 2022.
- Gonçalves et al., UFPR-ALPR, 2018.
- Hsieh et al., AOLP, 2011.