Spectra: Multi-Scale Implicit Decomposition for Deepfake Detection

Authors: Dhananjaya Jayasundara, Kartik Narayan, Vishal M. Patel

Published: 2026-10-22 22:36:28+00:00

AI Summary

Spectra is a deepfake detection framework that leverages multi-scale implicit neural representations (INRs) in conjunction with semantic visual features to improve generalization across unseen manipulation techniques and datasets. By analyzing the reconstruction dynamics of INRs, Spectra uncovers intrinsic differences between genuine and synthetic content that are not apparent in the image domain alone. This novel approach demonstrates superior robustness and cross-dataset generalization compared to existing methods, and its applicability extends to face presentation attack detection.

Abstract

Deepfakes increasingly undermine trust in digital media, yet existing detectors often struggle to generalize across unseen manipulation techniques, compression levels, and dataset shifts. Most approaches rely on RGB appearance, frequency-domain artifacts, or their combination, learning cues present in the final synthesized image. While effective in-distribution, such cues are frequently generator-specific and can diminish as synthesis models evolve. We argue that a complementary perspective can be obtained through implicit neural representations (INRs), which expose how an image is reconstructed rather than merely how it appears. By fitting an INR to an image and analyzing its internal activations, one can reveal reconstruction dynamics that are difficult to observe in the image domain alone and may capture more intrinsic differences between genuine and synthetic content. To this end, we propose Spectra, a deepfake detection framework that combines semantic visual representations with artifact-aware cues derived from INR reconstruction. Spectra extracts multi-level reconstruction heatmaps from image-specific INRs and jointly models them with semantic features through a bidirectional fusion mechanism. This enables direct interaction between image semantics and reconstruction artifacts, allowing the model to identify forgery cues that are less dependent on specific generation pipelines. Extensive experiments on multiple deepfake benchmarks demonstrate improved robustness and cross-dataset generalization, consistently outperforming detectors based solely on RGB appearance or semantic representations. Furthermore, we show that the proposed representation extends beyond deepfake detection to face presentation attack detection, highlighting its potential as a unified framework for facial attack detection.

Key findings

Spectra consistently outperforms state-of-the-art deepfake detectors on multiple out-of-distribution benchmarks, achieving the best average AUC (0.856). The INR-derived reconstruction representations provide complementary forensic evidence that significantly improves class separability. Furthermore, the framework demonstrates strong generalization to face presentation attack detection, highlighting its potential as a unified framework for facial authenticity tasks.

Approach

Spectra combines multi-level reconstruction heatmaps derived from image-specific implicit neural representations (INRs) with semantic features extracted by a frozen CLIP vision encoder. A bidirectional fusion mechanism allows these two representations to interact, enabling the model to identify forgery cues that are less dependent on specific generation pipelines. The fused features are then used for binary classification and supervised contrastive learning.

Datasets

FaceForensics++ (FF++) (c23 version) for training. Celeb-DF v1 (CDFv1), Celeb-DF v2 (CDFv2), FaceShifter (FSh), and Deep Fake Detection (DFD) for out-of-distribution evaluation. For presentation attack detection, CASIA-MFSD (C), Idiap Replay-Attack (I), MSU-MFSD (M), and OULU-NPU (O) datasets are used.

Model(s)

SIREN (for INR), CLIP vision transformer (ViT-L/14) (frozen), Hypernetwork (to predict SIREN parameters), MLP (for bottleneck projection and classification head).

Author countries

UNKNOWN