Redundant Semantic Environment Filling via Misleading-Learning for Fair Deepfake Detection

Authors: Xinan He, Yue Zhou, Shu Hu, Bin Li, Jiwu Huang, Feng Ding

Published: 2024-05-24 03:12:57+00:00

Comment: 19 pages, 9 figures

AI Summary

This paper introduces a novel misleading-learning strategy to counter the dual-overfitting problem in Deepfake detection, where models specialize in specific forgery fingerprints and demographic attributes, leading to unfair performance. By enriching the latent space with diverse redundant semantic environments, the approach effectively mitigates demographic bias while preserving high detection accuracy. Experimental results demonstrate superior fairness and generalization across various evaluation scenarios compared to existing state-of-the-art methods.

Abstract

Detecting falsified faces generated by Deepfake technology is essential for safeguarding trust in digital communication and protecting individuals. However, current detectors often suffer from a dual-overfitting: they become overly specialized in both specific forgery fingerprints and particular demographic attributes. Critically, most existing methods overlook the latter issue, which results in poor fairness: faces from certain demographic groups, such as different genders or ethnicities, are consequently more difficult to reliably detect. To address this challenge, we propose a novel strategy called misleading-learning, which populates the latent space with a multitude of redundant environments. By exposing the detector to a sufficiently rich and balanced variety of high-level information for demographic fairness, our approach mitigates demographic bias while maintaining a high detection performance level. We conduct extensive evaluations on fairness, intra-domain detection, cross-domain generalization, and robustness. Experimental results demonstrate that our framework achieves superior fairness and generalization compared to state-of-the-art approaches.

Key findings

The framework achieves superior fairness by significantly reducing the detection performance gap among different demographic subgroups, while also maintaining high overall detection accuracy and strong generalization capabilities across intra-domain and cross-domain datasets. Furthermore, the method exhibits notable robustness against various video post-processing disturbances.

Approach

The proposed misleading-learning strategy addresses demographic bias by populating the latent space with redundant semantic environments. It involves selecting demographically diverse samples to 'mislead' the detector, thereby compelling it to reduce reliance on specific demographic attributes. An Astray-SRM filter is also integrated for coarse screening of high-level semantic information before feature extraction and fusion.

Datasets

FaceForensics++ (FF++), DeepFakeDetection (DFD), Deepfake Detection Challenge (DFDC), Celeb-DF

Model(s)

The proposed method utilizes an adapter structure with various backbones, including Xception, ResNet-34, ResNet-50, and EfficientNet-b4. It incorporates a Redundant Semantic Feature Extractor (Ered) leveraging zero-shot pretrained weights and a Forgery Feature Discriminator (Dsub).

Author countries

China, USA

← Previous