Mask interpolation - definition
Mask interpolation is the process of generating intermediate privacy masks between manually defined keyframes in an image sequence or video. A privacy mask is a shape that marks pixels to be blurred, pixelated, covered, or otherwise obscured. In video anonymization, an editor may place masks on a face, license plate, name badge, document, or screen in selected frames. The software then estimates the mask position, size, shape, or rotation for the frames between those keyframes.
Mask interpolation reduces the amount of frame-by-frame manual editing required when an object moves predictably. It is not the same as image interpolation, video frame interpolation, object detection, or facial recognition. Its output is mask geometry and timing, not new video content and not an identity decision. The resulting masks are then applied to the original pixels to obscure the selected visual information.
For face and license plate blurring, interpolation is commonly used after automatic face detection or license plate detection has missed an object, lost tracking, or produced an incomplete mask. It is also used for visual elements that are not automatically detected, such as documents, name badges, tattoos, company logos, or content displayed on monitors. These elements require manual masking in Gallio PRO.
How mask interpolation works in video anonymization
An editor first creates keyframes. Each keyframe contains a mask definition at a specific video frame or timestamp. The definition can include a rectangle, ellipse, polygon, or other supported shape, together with its coordinates and blur settings. The interpolation method calculates mask values for the frames between two keyframes.
For a mask parameter p, such as horizontal position, vertical position, width, height, or rotation, simple linear interpolation can be expressed as:
p(t) = p0 + ((t - t0) / (t1 - t0)) × (p1 - p0)
Here, t0 and t1 are the two keyframe times. Linear interpolation assumes that the parameter changes at a constant rate between the keyframes. This can be adequate when a person or vehicle moves smoothly across the frame. It can fail when movement is abrupt, curved, occluded, or affected by camera motion.
Interpolation approach | Mask behavior | Typical use | Main limitation
|
|---|---|---|---|
Linear | Moves or resizes evenly between keyframes | Stable camera and predictable movement | Does not model acceleration or curved paths |
Bezier or spline | Uses a curved transition between keyframes | Smoother manually controlled movement | Can overshoot the object boundary if poorly configured |
Optical-flow-assisted | Uses estimated pixel motion to propagate a mask | Detailed movement with visible texture | Can fail during occlusion, blur, reflections, or scene cuts |
Object tracking-assisted | Updates a mask from a tracked object location | Faces or license plates across longer sequences | Requires review when tracking confidence drops |
Key parameters for mask interpolation
Interpolation quality should be assessed as a privacy-control issue, not only as an editing convenience. A mask that is visually smooth but exposes part of a face or license plate is not sufficient. The reviewer should inspect the full interval between keyframes, including the frames immediately before and after each keyframe.
- Keyframe spacing: The number of frames between manually verified masks. Shorter intervals generally provide more control when motion changes rapidly.
- Mask margin: The additional area around the detected or manually selected object. A small margin can expose pixels at the boundary. An excessive margin can obscure unrelated people, vehicles, or scene content.
- Scale change: The change in mask width and height as a face or license plate approaches or moves away from the camera.
- Rotation and perspective: A rectangular license plate can appear trapezoidal or rotate relative to the camera. A fixed horizontal rectangle may not cover it adequately.
- Occlusion state: The mask may need to remain active when an object is partly hidden by another person, vehicle, or physical object.
- Scene-cut detection: Masks should not be interpolated across a cut to a different shot. A new shot requires separate review and usually new keyframes.
Practical workflow for manual mask interpolation
A controlled workflow combines interpolation with visual quality assurance. The purpose is to avoid relying on estimated mask positions where the underlying footage changes in ways that the interpolation model cannot represent.
- Review the video at normal speed and identify faces, license plates, and manually selected visual information that requires masking.
- Create a keyframe at the first frame where the object becomes visible.
- Create another keyframe before a change in direction, scale, pose, occlusion, camera movement, or scene cut.
- Apply interpolation only within a visually continuous segment.
- Inspect intermediate frames at playback speed and frame-by-frame around rapid movement.
- Correct mask position, size, or shape by adding keyframes rather than accepting exposed pixels.
- Verify the exported video, because rendering, cropping, scaling, and transcoding can change the visible mask boundary.
Limitations and common failure modes
Mask interpolation cannot guarantee anonymization by itself. It estimates a transition between known states. If the object moves unpredictably, becomes motion-blurred, changes orientation, or disappears behind another object, the estimated mask can drift. Drift can leave identifiable pixels visible for only a few frames, which may still be sufficient for viewing, extraction, or re-identification.
Optical flow is one method for estimating motion between frames. The foundational formulation by Horn and Schunck describes optical flow as an estimation problem based on brightness changes and spatial smoothness assumptions. Those assumptions are weakened by lighting variation, reflections, compression artifacts, and motion blur. See Horn and Schunck, “Determining Optical Flow” (1981).
For privacy editing, a conservative mask margin and additional keyframes are usually more reliable than attempting to preserve a tight mask boundary during difficult motion. This is particularly important for small, distant, angled, or partially occluded license plates.
Use cases for mask interpolation
Mask interpolation is useful when video must be prepared for internal review, publication, training, incident analysis, research, or sharing with a third party. It supports both automatically detected objects and objects selected manually by an editor.
- Extending a face blur through frames where face detection temporarily loses the face.
- Maintaining a license plate blur while a vehicle moves through a fixed-camera scene.
- Masking a document, monitor, logo, tattoo, or name badge that is visible across several frames.
- Correcting a mask after a person turns their head, enters shadow, or is partly covered by another person.
- Creating consistent masking across a sequence before exporting an anonymized video.
Standards and references
Mask interpolation is an editing and computer-vision technique rather than a separately standardized legal or anonymization method. Its implementation should nevertheless be documented within a repeatable privacy-processing workflow. The National Institute of Standards and Technology Artificial Intelligence Risk Management Framework 1.0 (NIST AI RMF 1.0, 2023) identifies validation, monitoring, and documented risk management as relevant controls for artificial intelligence-enabled systems.
For video coding and frame timing, the International Organization for Standardization and International Electrotechnical Commission publish the ISO/IEC 14496 family of Moving Picture Experts Group standards. Video editors should preserve source frame timing and verify the rendered output because frame-rate conversion or transcoding can alter the relationship between keyframes and visible frames.