Cross-Domain Local Characteristic Enhanced Deepfake Video Detection

Authors: Zihan Liu, Hanyi Wang, Shilin Wang

Published: 2022-11-07 07:44:09+00:00

AI Summary

This paper introduces Cross-Domain Local Forensics (XDLF), a novel pipeline for general deepfake video detection designed to overcome poor generalization to unseen manipulations. XDLF simultaneously exploits local forgery patterns from space, frequency, and time domains. It leverages four forgery-sensitive local facial regions to guide the model in enhancing subtle artifacts and localizing potential anomalies, demonstrating impressive performance, especially in cross-dataset generalization.

Abstract

As ultra-realistic face forgery techniques emerge, deepfake detection has attracted increasing attention due to security concerns. Many detectors cannot achieve accurate results when detecting unseen manipulations despite excellent performance on known forgeries. In this paper, we are motivated by the observation that the discrepancies between real and fake videos are extremely subtle and localized, and inconsistencies or irregularities can exist in some critical facial regions across various information domains. To this end, we propose a novel pipeline, Cross-Domain Local Forensics (XDLF), for more general deepfake video detection. In the proposed pipeline, a specialized framework is presented to simultaneously exploit local forgery patterns from space, frequency, and time domains, thus learning cross-domain features to detect forgeries. Moreover, the framework leverages four high-level forgery-sensitive local regions of a human face to guide the model to enhance subtle artifacts and localize potential anomalies. Extensive experiments on several benchmark datasets demonstrate the impressive performance of our method, and we achieve superiority over several state-of-the-art methods on cross-dataset generalization. We also examined the factors that contribute to its performance through ablations, which suggests that exploiting cross-domain local characteristics is a noteworthy direction for developing more general deepfake detectors.

Key findings

The proposed XDLF method achieved superior performance on cross-dataset generalization, outperforming state-of-the-art methods on Celeb-DF and DFDC datasets. Ablation studies confirmed that the FSLR-Guided Feature Enhancement, Cross Attention, and Feature Fusion modules are all essential for the model's strong performance and improved generalization. The findings highlight the importance of exploiting cross-domain local characteristics for developing more general deepfake detectors.

Approach

The authors propose Cross-Domain Local Forensics (XDLF), a two-stream 3D CNN based framework. It simultaneously extracts features from RGB images (space-time domain) and adaptively generated frequency maps (frequency-time domain), fusing them using Cross Attention and Feature Fusion modules. A Forgery-Sensitive Local Region (FSLR)-Guided Feature Enhancement module further guides the model to focus on subtle artifacts in four critical facial regions (eyes, nose, mouth).

Datasets

FaceForensics++ (FF++), Celeb-DF (CDF), DeepFake Detection Challenge (DFDC)

Model(s)

3D ResNet-50 (as backbone for two-stream 3D CNNs), with custom modules for Frequency Decomposition, Forgery-Sensitive Region Detection, FSLR-Guided Feature Enhancement (FGFE), Cross Attention, and Feature Fusion.

Author countries

China

← Previous