CAM-VFD: Cross-Attention Multimodal Video Forgery Detection

Authors: Hoda Osama Elkhodary, Sherin Mostafa Youssef, Marwa Elshenawy, Dalia Sobhy

Published: 2026-05-16 19:46:33+00:00

AI Summary

CAM-VFD is a Cross-Attention Multimodal Video Forgery Detection framework that addresses the limitations of single-modality detectors by identifying cross-modal contradictions in appearance, motion, and depth features. It proposes modeling these inconsistencies as a directional forensic signal, significantly improving deepfake detection robustness and generalization to unseen generative models and real-world degradations. The framework leverages CLIP, VideoMAE, and MiDaS features with a cross-attention fusion mechanism.

Abstract

The rapid advancement of Deepfake technologies and video manipulation tools poses a critical challenge to multimedia forensics, judicial evidence integrity, and information authenticity. Current detectors rely on single-modality signals, treating appearance, geometry, and motion independently. However, advanced generators maintain within-modality consistency while producing cross-modal contradictions, which are forensically discriminative but invisible to any single-modal detector. We propose CAM-VFD, a Cross-Attention Multimodal Video Forgery Detection framework that models cross-modal contradiction as a directional forensic signal. The framework uses a cross-attention fusion mechanism in which CLIP-based appearance representations serve as queries against VideoMAE motion features and MiDaS depth features, enabling the identification of contradictions between visual, temporal, and geometric evidence. We examine this design through cross-modal attention discrepancy analysis, observing statistically separable real and fake distributions ($p<0.001$, Cohen's $d=0.68$). Experimental results on two generative video benchmarks indicate consistent performance, with 95.31\\% Top-1 accuracy on GenVidBench and 93.43\\% accuracy, 90.63\\% F1-score, and 96.56\\% AUROC on GenVideo. Moreover, CAM-VFD demonstrates stable performance under compression, noise, blur, and adversarial perturbations, suggesting that cross-modal reasoning may improve robustness in media forensics. The code is publicly available at \\url{https://github.com/Hoda-Osama/CAM-VFD/tree/main}.

Key findings

CAM-VFD achieved high performance, with 95.31% Top-1 accuracy on GenVidBench and 93.43% accuracy, 90.63% F1-score, and 96.56% AUROC on GenVideo, consistently outperforming state-of-the-art baselines. The model demonstrated stable performance under various real-world degradations (compression, noise, blur) and adversarial perturbations. A Cross-Modal Attention Discrepancy analysis confirmed statistically separable distributions for real and fake videos (p<0.001, Cohen's d=0.68), validating the cross-modal contradiction premise.

Approach

The proposed CAM-VFD framework adaptively samples video frames and extracts features for appearance, motion, and depth using pre-trained CLIP-ViT B/32, VideoMAE-Base, and MiDaS DPT-Hybrid encoders, respectively. A cross-attention fusion mechanism then uses appearance representations as queries against motion and depth features to identify cross-modal contradictions, which are subsequently fed into an MLP classifier for binary deepfake detection.

Datasets

GenVidBench, GenVideo

Model(s)

UNKNOWN

Author countries

Egypt

← Previous