Compressed Deepfake Video Detection Based on 3D Spatiotemporal Trajectories

Authors: Zongmei Chen, Xin Liao, Xiaoshuai Wu, Yanxiang Chen

Published: 2024-04-28 11:48:13+00:00

AI Summary

This paper proposes a novel deepfake video detection method specifically designed for compressed videos, addressing a common limitation of existing techniques. It leverages 3D spatiotemporal trajectories by constructing robust motion features from a 3D model, integrating 2D and 3D frame details, and decoupling facial expressions from head movements. A sequential analysis method based on phase space motion trajectories is then used to differentiate between genuine and fake faces, yielding strong performance on compressed deepfake benchmarks.

Abstract

The misuse of deepfake technology by malicious actors poses a potential threat to nations, societies, and individuals. However, existing methods for detecting deepfakes primarily focus on uncompressed videos, such as noise characteristics, local textures, or frequency statistics. When applied to compressed videos, these methods experience a decrease in detection performance and are less suitable for real-world scenarios. In this paper, we propose a deepfake video detection method based on 3D spatiotemporal trajectories. Specifically, we utilize a robust 3D model to construct spatiotemporal motion features, integrating feature details from both 2D and 3D frames to mitigate the influence of large head rotation angles or insufficient lighting within frames. Furthermore, we separate facial expressions from head movements and design a sequential analysis method based on phase space motion trajectories to explore the feature differences between genuine and fake faces in deepfake videos. We conduct extensive experiments to validate the performance of our proposed method on several compressed deepfake benchmarks. The robustness of the well-designed features is verified by calculating the consistent distribution of facial landmarks before and after video compression.Our method yields satisfactory results and showcases its potential for practical applications.

Key findings

The proposed method achieves state-of-the-art detection accuracy on compressed deepfake videos, outperforming existing methods like TALL-Swim on FF++LQ and BRCNet on DFDC. It also exhibits high detection efficiency with minimal training and testing times, making it suitable for practical applications. The robustness of the approach stems from its reliance on facial landmark distributions, which are shown to be largely unaffected by video compression.

Approach

The method constructs spatiotemporal motion features using a robust 3D model for facial landmark localization and tracking, explicitly separating facial expressions from head movements. These features are then processed using time-delayed embedding to reconstruct phase space trajectories. A lightweight Transformer architecture performs sequential analysis on these trajectories to explore feature differences and classify videos.

Datasets

FaceForensics++ (FF++), DFDC, Celeb-DF

Model(s)

3D morphable face model (3DMM) for feature construction, Dlib for landmark localization, and a lightweight Transformer architecture for classification.

Author countries

China

← Previous