Deepfake Detection Generalization with Diffusion Noise

Authors: Hongyuan Qi, Wenjin Hou, Hehe Fan, Jun Xiao

Published: 2026-04-16 03:02:04+00:00

Comment: 17 pages

AI Summary

This paper introduces Attention-guided Noise Learning (ANL), a novel framework designed to enhance deepfake detection generalization, particularly for content generated by diffusion models. ANL integrates a pre-trained diffusion model into the detection pipeline to leverage subtle diffusion noise characteristics, guiding a forensic classifier to learn more robust and globally distributed features through an attention mechanism. Extensive experiments demonstrate that ANL significantly outperforms existing methods, achieving state-of-the-art accuracy and strong generalization to unseen forgery types and generative models without additional inference overhead.

Abstract

Deepfake detectors face growing challenges in generalization as new image synthesis techniques emerge. In particular, deepfakes generated by diffusion models are highly photorealistic and often evade detectors trained on GAN-based forgeries. This paper addresses the generalization problem in deepfake detection by leveraging diffusion noise characteristics. We propose an Attention-guided Noise Learning (ANL) framework that integrates a pre-trained diffusion model into the deepfake detection pipeline to guide the learning of more robust features. Specifically, our method uses the diffusion model's denoising process to expose subtle artifacts: the detector is trained to predict the noise contained in an input image at a given diffusion step, forcing it to capture discrepancies between real and synthetic images, while an attention-guided mechanism derived from the predicted noise is introduced to encourage the model to focus on globally distributed discrepancies rather than local patterns. By harnessing the frozen diffusion model's learned distribution of natural images, the ANL method acts as a form of regularization, improving the detector's generalization to unseen forgery types. Extensive experiments demonstrate that ANL significantly outperforms existing methods on multiple benchmarks, achieving state-of-the-art accuracy in detecting diffusion-generated deepfakes. Notably, the proposed framework boosts generalization performance (e.g., improving ACC/AP by a substantial margin on unseen models) without introducing additional overhead during inference. Our results highlight that diffusion noise provides a powerful signal for generalizable deepfake detection.

Key findings

The ANL framework significantly outperforms existing state-of-the-art methods across multiple benchmarks, achieving substantial improvements in accuracy and average precision (e.g., over 12% ACC gain on DiffFace in cross-model evaluation). It demonstrates strong generalization capabilities to unseen generative models and diverse datasets, attributable to its focus on fundamental diffusion noise characteristics. The method also operates without introducing additional overhead during inference, making it practical for real-world deployment.

Approach

The Attention-guided Noise Learning (ANL) framework utilizes a pre-trained diffusion model's denoising process to estimate the noise component within an input image at a single timestep. A spatial attention map is then constructed from this predicted noise, reflecting its intensity distribution. This attention map subsequently guides a ResNet-based classification network to focus on globally distributed noise discrepancies, enabling robust classification of real versus diffusion-generated deepfakes.

Datasets

DiffFace, DiFF, DiffusionForensics

Model(s)

ADM (for noise estimation), ResNet-50 (as the forensic classifier backbone)

Author countries

China

← Previous