GRACE: Graph-Regularized Attentive Convolutional Entanglement with Laplacian Smoothing for Robust DeepFake Video Detection

Authors: Chih-Chung Hsu, Shao-Ning Chen, Mei-Hsuan Wu, Yi-Fang Wang, Chia-Ming Lee, Yi-Shiuan Chou

Published: 2024-06-28 14:17:16+00:00

Comment: Submitted to TPAMI 2024

AI Summary

This paper introduces GRACE (Graph-Regularized Attentive Convolutional Entanglement) for robust DeepFake video detection, specifically addressing performance degradation caused by noisy face sequences from degraded videos or adversarial attacks. GRACE leverages a graph convolutional network with graph Laplacian smoothing and sparsity constraints to entangle spatiotemporal features, effectively suppressing noise and enhancing detection stability. Comprehensive experiments demonstrate state-of-the-art performance under challenging noisy conditions.

Abstract

As DeepFake video manipulation techniques escalate, posing profound threats, the urgent need to develop efficient detection strategies is underscored. However, one particular issue lies with facial images being mis-detected, often originating from degraded videos or adversarial attacks, leading to unexpected temporal artifacts that can undermine the efficacy of DeepFake video detection techniques. This paper introduces a novel method for robust DeepFake video detection, harnessing the power of the proposed Graph-Regularized Attentive Convolutional Entanglement (GRACE) based on the graph convolutional network with graph Laplacian to address the aforementioned challenges. First, conventional Convolution Neural Networks are deployed to perform spatiotemporal features for the entire video. Then, the spatial and temporal features are mutually entangled by constructing a graph with sparse constraint, enforcing essential features of valid face images in the noisy face sequences remaining, thus augmenting stability and performance for DeepFake video detection. Furthermore, the Graph Laplacian prior is proposed in the graph convolutional network to remove the noise pattern in the feature space to further improve the performance. Comprehensive experiments are conducted to illustrate that our proposed method delivers state-of-the-art performance in DeepFake video detection under noisy face sequences. The source code is available at https://github.com/ming053l/GRACE.

Key findings

GRACE achieves state-of-the-art performance in DeepFake video detection, notably exhibiting superior robustness and reliability under highly noisy face sequences (high masking ratios) and adversarial attacks on face detectors. It significantly outperforms other methods in these challenging scenarios, which typically suffer substantial performance degradation. The method effectively mitigates the impact of invalid facial images and adversarial noise, providing a generalized and robust solution for real-world DeepFake detection.

Approach

The GRACE method first employs a conventional CNN backbone to extract spatiotemporal features from video frames. It then constructs an affinity matrix through Feature Entanglement to mutually intertwine these spatial and temporal features, forming a graph representation. This graph is processed by a Graph Convolutional Network (GCN) with Graph Laplacian Smoothing Prior Regularization (GLSPR) to filter out noise, augmented by a Sparsity Constraint (SC) to prioritize essential features of valid faces, enabling robust DeepFake detection.

Datasets

FF++ [1], Celeb-DFv2 [3], DFDC [2]

Model(s)

UNKNOWN

Author countries

Taiwan

← Previous