Preserving Knowledge across Space and Time for Continual Video Deepfake Detection

Authors: Taehoon Kim, Jongwook Choi, Heejae Jo, Byungmin Park, Jongwon Choi

Published: 2026-09-03 07:02:51+00:00

Comment: Accepted by ECCV 2025. Code will be available at github.com/rama0126/MSFD

AI Summary

This paper introduces Modality-Specific Frequency Distillation (MSFD), a continual deepfake video detection framework that addresses the challenge of adapting to new forgery patterns while preserving knowledge of old ones. MSFD explicitly decomposes video features into spatial, temporal, and spatiotemporal modalities in the frequency domain, enabling independent preservation of each. It also incorporates a cross-modality decorrelation loss to ensure distinct spatiotemporal representations.

Abstract

The continuous emergence of high-quality video deepfakes requires detectors that continually adapt to new forgery patterns, yet existing approaches, which are designed for deepfake images, fail to capture video-specific cues. Unlike deepfake images that contain only spatial artifacts, deepfake videos leave distinct evidence along both spatial and temporal axes, necessitating the separate preservation of each modality during sequential model updates. To overcome this limitation, we introduce a continual deepfake video detection framework, Modality-Specific Frequency Distillation (MSFD), that explicitly decomposes video features into spatial, temporal, and spatiotemporal modalities in the frequency domain. This decomposition enables independent preservation of each modality, as different deepfake video types exhibit varying reliance on spatial and temporal cues across tasks. Furthermore, MSFD adopts a cross-modality decorrelation loss that encourages spatiotemporal representations to remain orthogonal to single-modality cues. Extensive experiments show that our framework achieves stronger adaptation and preserves performance more effectively than state-of-the-art methods across diverse continual deepfake video scenarios.

Key findings

The MSFD framework consistently outperforms state-of-the-art methods in various continual deepfake video scenarios, achieving higher overall AUC and lower forgetting. It demonstrates stronger adaptability and better preservation of performance, particularly with limited memory. The modality-specific frequency decomposition and decorrelation loss are crucial for effectively handling diverse deepfake types and ensuring robustness against perturbations.

Approach

MSFD decomposes intermediate video features into spatial, temporal, and spatiotemporal frequency spectra using Fast Fourier Transforms. It then uses a Modality-Specific Adaptor (MSA) for reweighting and distillation masking of these spectra, and a Cross-Modality Decorrelation Loss (CDL) to encourage orthogonal spatiotemporal representations. These components are integrated with standard classification and logit distillation losses for continual learning.

Datasets

FF++, DFD, CDF, DFDCp, FFIW, KoDF, FSh, DFo, DF40, AIGVDBench

Model(s)

3D ResNet-18, ViT-B/16 (CLIP), Xception + Transformer

Author countries

Republic of Korea

← Previous