On Improving Robustness of Deepfake Image Detectors

Authors: Abu Taib Mohammed Shahjahan, Mohammad Mannan, Abdessamad Ben Hamza, Amr Youssef

Published: 2026-06-01 19:03:32+00:00

Comment: Accepted at Usenix Security 2026

AI Summary

This paper proposes a unified framework to enhance the robustness of deepfake image detectors against adversarial attacks without relying on adversarial training data. The framework integrates higher-order statistical modeling in the frequency domain (DCT-based moment pooling up to fourth order), content-agnostic noise residual features, and cross-scene generalization through patch-level semantic disruption. This approach leverages the insight that adversarial attacks often overlook higher-order residual-frequency characteristics, particularly kurtosis, significantly improving detector performance.

Abstract

The rapid advancement of Generative AI has introduced remarkable opportunities while simultaneously raising critical concerns regarding content authenticity. While recent work has increasingly focused on improving the generalization of deepfake detectors across unseen generative models, their robustness against adversarial attacks remains limited. In particular, Abdullah et al. (IEEE SP 2024) evaluated eight detectors and demonstrated that most of them exhibit significant performance degradation under adversarial attacks. We also observed the same phenomenon by testing seven most recent state-of-the-art detectors. To address this problem, we propose a unified framework that integrates three complementary design principles without relying on adversarial training data: (i) higher-order statistical modeling in the frequency domain via Discrete Cosine Transform (DCT)-based moment pooling up to fourth order, (ii) content-agnostic feature representations derived from noise residuals, and (iii) cross-scene generalization enforced through patch-level semantic disruption. A key insight underpinning our approach is that adversarial attacks primarily operate on low-order statistics and visual semantics, leaving higher-order residual-frequency characteristics, particularly kurtosis, largely unconstrained. Extensive experiments demonstrate that our method consistently improves robustness across six architecturally diverse detectors. Notably, we achieve up to 88.9% reduction in recall degradation on current adversarial benchmarks, and improve the best-performing recent detector (Yang et al., IEEE CVPR 2025) from 81.9% to 97.15% accuracy under attack. Overall, our method provides a principled, architecture-agnostic approach for improving deepfake detection robustness against current attacks.

Key findings

The method consistently improved robustness across six architecturally diverse detectors. It achieved up to an 88.9% reduction in recall degradation on current adversarial benchmarks. Notably, it improved the accuracy of the best-performing recent detector (D3) from 81.9% to 97.15% under attack.

Approach

The authors propose a unified framework integrating three design principles: higher-order statistical modeling using DCT-based moment pooling up to fourth order in the frequency domain, content-agnostic features derived from MM-BSN noise residuals, and cross-scene generalization enforced by patch-level semantic disruption (shuffling and rotation). This method is designed to be integrated into existing detectors, exploiting statistical blind spots of adversarial attacks.

Datasets

GenImage, StyleCLIP (non-adversarial subset for training from Abdullah et al.), and various adversarial datasets from Abdullah et al. [1] (including Surrogate StyleCLIP, AdvImages w/ Surrogate, Advtrained CLIPResNet advV2, etc.) for testing.

Model(s)

The proposed framework utilizes CLIP ViT-L/14 as a backbone and MM-BSN for denoising. It is integrated and tested with six existing deepfake detectors: D3, DCT, DE-FAKE, CNN-F, Patch-Forensics, and Resynthesis.

Author countries

Canada

← Previous