Towards Robust Protective Perturbation against DeepFake Face Swapping

View on arXiv ← Back to list

Authors: Hengyang Yao, Lin Li, Ke Sun, Jianing Qiu, Huiping Chen

Published: 2025-12-08 07:12:43+00:00

AI Summary

This paper addresses the fragility of proactive DeepFake defenses, which use invisible perturbations to disrupt face swapping, against common image transformations. The authors propose Expectation Over Learned distribution of Transformation (EOLT), a reinforcement learning framework that treats the transformation distribution as a learnable component. EOLT uses a policy network to adaptively prioritize critical transformations, resulting in instance-specific perturbations that achieve substantially higher robustness than standard uniform sampling methods.

Abstract

DeepFake face swapping enables highly realistic identity forgeries, posing serious privacy and security risks. A common defence embeds invisible perturbations into images, but these are fragile and often destroyed by basic transformations such as compression or resizing. In this paper, we first conduct a systematic analysis of 30 transformations across six categories and show that protection robustness is highly sensitive to the choice of training transformations, making the standard Expectation over Transformation (EOT) with uniform sampling fundamentally suboptimal. Motivated by this, we propose Expectation Over Learned distribution of Transformation (EOLT), the framework to treat transformation distribution as a learnable component rather than a fixed design choice. Specifically, EOLT employs a policy network that learns to automatically prioritize critical transformations and adaptively generate instance-specific perturbations via reinforcement learning, enabling explicit modeling of defensive bottlenecks while maintaining broad transferability. Extensive experiments demonstrate that our method achieves substantial improvements over state-of-the-art approaches, with 26% higher average robustness and up to 30% gains on challenging transformation categories.

Key findings

The study revealed that transformations vary widely in their generalization behavior, confirming that standard Expectation Over Transformation (EOT) with uniform sampling is suboptimal. EOLT achieved 26% higher average robustness compared to state-of-the-art approaches and up to 30% gains on challenging transformation categories like Stylization. The learned policy successfully prioritized beneficial transformations (e.g., blur and noise) that enhance transferability while downweighting toxic transformations, demonstrating strong generalization to unseen data and unseen transformation categories.

Approach

The core approach involves training a policy network using reinforcement learning (REINFORCE) to learn an optimal, non-uniform probability distribution over 30 predefined sub-policies (transformation-intensity combinations). This learned distribution is used within the Projected Gradient Descent (PGD) optimization loop (replacing the uniform sampling of EOT) to generate robust protective perturbations. The objective is to maximize the disruption effect of the perturbation under a set of validation transformations.

Datasets

FFHQ

Model(s)

PGD, PreAct ResNet-18 (Policy Model Backbone), SimSwap, ReFace (Target DeepFake Models)

Author countries

UK, China, UAE

← Previous