Image Segmentation - Definition
Image segmentation is the process of assigning labels to individual pixels in an image or video frame in order to isolate regions that belong to specific objects or classes. The main types include semantic segmentation (all pixels of a given class, e.g., a face), instance segmentation (separate masks for each object instance, e.g., multiple faces), and panoptic segmentation (a combination of both approaches). A major leap in performance was achieved with convolutional neural networks and architectures such as FCN and U-Net, followed by detection-segmentation methods like Mask R-CNN, and more recent transformer-based and interactive segmentation models.
In the context of photo and video anonymization, image segmentation is used to generate precise pixel-level masks for sensitive objects. These masks serve as the basis for applying face blurring, license plate blurring, pixelation, or other redaction techniques, minimizing the risk of identity disclosure while preserving the overall context of the scene.
The Role of Image Segmentation in Face and License Plate Anonymization
In practice, video anonymization and image redaction combine object detection with image segmentation. The detector identifies faces and license plates, while instance segmentation refines the exact shape of each object. This ensures that the blur effect does not cover unnecessary background areas and does not leave unmasked gaps along object boundaries.
For video sequences, masks are propagated across frames using object tracking and spatiotemporal optimization, reducing flicker and preventing object loss between frames.
From a GDPR compliance perspective, masking faces and visual identifiers supports the principles of data minimization and integrity and confidentiality (Article 5(1)(c) and (f) of Regulation (EU) 2016/679). The European Data Protection Board, in Guidelines 3/2019 on video devices, recommends applying measures that limit the scope of processing, including masking or obscuring sensitive areas. In Western European countries, rules regarding the publication or sharing of materials containing visible license plates stem from national regulations and data protection authority practices rather than from a single uniform EU-wide requirement. In Poland, interpretations vary; however, guidance from the Polish Data Protection Authority (UODO) and EDPB positions emphasize reducing identifiability in published materials.
In Gallio PRO, image segmentation is used to generate masks for faces and license plates as a preparatory step before blurring. The software automatically covers these two categories and does not perform real-time anonymization or live video stream processing. Other elements, such as logos or tattoos, can be manually masked within the editor.
Segmentation Technologies and Models for Privacy Protection
Deep learning-based methods are primarily used for image segmentation in anonymization because they provide high-quality masks under complex lighting and motion conditions. Traditional techniques such as graph cuts or superpixels are now rarely used independently but may still serve as post-processing components.
- FCN and U-Net - End-to-end architectures for semantic segmentation. U-Net is particularly useful with limited datasets due to skip connections and strong data augmentation.
- Mask R-CNN - Extends object detection with a mask head, enabling independent masking of multiple objects in a single scene, including multiple faces.
- DeepLabv3+ and related models - Use atrous convolutions and decoders to improve object boundary accuracy.
- Transformer-based segmentation and interactive tools - Including models such as SegFormer and prompt-based approaches. Models like Segment Anything enable rapid mask generation from coarse user prompts (points or bounding boxes).
Training segmentation models for anonymization requires datasets annotated with pixel-level masks for faces and license plates. In practice, instance segmentation datasets are used, or detection models are combined with shape refinement techniques. Data sources include COCO and Open Images for instance segmentation, and datasets such as CelebAMask-HQ for face segmentation. For license plates, detection datasets and proprietary instance annotations are commonly used, ensuring that the annotation process remains legally compliant.
Key Parameters and Segmentation Metrics
Segmentation model selection and evaluation should rely on measurable performance indicators. In anonymization workflows, mask quality metrics and computational performance parameters are particularly important, as they directly impact processing time and privacy risk.
Metric / Parameter | Definition | Formula or Description | Source
|
|---|---|---|---|
IoU (Intersection over Union) | Overlap between predicted and ground-truth masks | |A ∩ B| / |A ∪ B| | COCO evaluation protocol, Lin et al., 2014 |
mIoU | Mean IoU across classes | Average IoU computed per class | Cityscapes; Cordts et al., 2016 |
Dice (Pixel-level F1 score) | Similarity metric emphasizing overlap | 2|A ∩ B| / (|A| + |B|) | Sørensen-Dice; see Ronneberger et al., 2015 |
APmask | Average Precision for masks at IoU thresholds | Mean precision for IoU ∈ [0.5, 0.95] | COCO, Lin et al., 2014 |
Latency | Processing time per image/frame | Milliseconds per image or frame | Deployment specifications |
Throughput | System processing capacity | FPS (frames per second) | Deployment specifications |
Memory Usage | GPU/CPU memory consumption during inference | MB or GB | Deployment specifications |
FN Rate | Proportion of undetected class pixels | 1 − pixel recall | Comparative studies |
In anonymization scenarios, minimizing false negatives at object boundaries is critical. Techniques such as mask dilation and edge filtering are commonly applied to reduce blur leakage and ensure complete coverage of sensitive regions.
Challenges and Limitations in Compliance Applications
Effective image segmentation in data protection projects faces both technical and regulatory challenges. Below are the most common issues and their impact on anonymization quality.
- Variable imaging conditions - Low contrast, nighttime scenes, motion blur, and visual noise affect mask boundaries.
- Domain variability - Differences in camera types, countries, license plate formats, and facial coverings reduce performance outside the training domain.
- False negative risk - Unblurred facial fragments or license plate characters may violate privacy regulations, including GDPR.
- Computational resources - Instance segmentation is resource-intensive. For sensitive data, on-premise processing is often preferred, limiting cloud transfer.
- Data governance - Storing masks and metadata should minimize re-identification risk. Gallio PRO does not collect logs containing face or license plate detection data.
- Automation limits - In Gallio PRO, automatic detection is limited to faces and license plates. Other elements must be manually masked in the editor.
Normative References and Sources
The following documents and publications underpin the definitions, metrics, and recommendations related to image segmentation and visual data processing in privacy contexts.
- Regulation (EU) 2016/679 of the European Parliament and of the Council (GDPR).
- European Data Protection Board, Guidelines 3/2019 on processing of personal data through video devices (adopted after public consultation on 29 January 2020).
- ISO/IEC 27001:2022 - Information security, cybersecurity and privacy protection - Information security management systems.
- ISO/IEC 23894:2023 - Information technology - Artificial intelligence - Risk management.
- Long, Shelhamer, Darrell, Fully Convolutional Networks for Semantic Segmentation, CVPR 2015.
- Ronneberger, Fischer, Brox, U-Net: Convolutional Networks for Biomedical Image Segmentation, MICCAI 2015.
- He, Gkioxari, Dollár, Girshick, Mask R-CNN, ICCV 2017.
- Kirillov et al., Panoptic Segmentation, CVPR 2019.
- Lin et al., Microsoft COCO: Common Objects in Context, ECCV 2014.
- Cordts et al., The Cityscapes Dataset for Semantic Urban Scene Understanding, CVPR 2016.
- Kuznetsova et al., The Open Images Dataset V6, IJCV 2020.
- Lee et al., CelebAMask-HQ: A Large-Scale Face Parsing Dataset, CVPR Workshops 2020.
- Kirillov et al., Segment Anything, arXiv 2023.