Adaptive Compression and Targeted Perturbation: A Unified Framework for Generalized Audio Deepfake Detection

Authors: Jianqiao Cui, Haoxian Zhu, Hanbiao Meng, Yanghao Zhang

Published: 2026-10-12 01:40:48+00:00

Comment: 10 pages,1 figure, Accepted by NeurIPS 2026

AI Summary

This paper introduces a unified framework for audio deepfake detection (ADD) that enhances generalization and robustness by integrating two key modules. The Adaptive Audio Feature Integration (AAFI) module dynamically selects optimal latent dimensions, while the Mel-Band Adversarial Perturbation (Mel-BAP) strategy applies targeted regularization in the mel-spectrogram domain. This approach achieves state-of-the-art performance with significantly fewer parameters compared to existing models, demonstrating strong zero-shot transferability and resilience to real-world acoustic distortions.

Abstract

Existing audio deepfake detection (ADD) methods frequently exhibit limited generalization, as fixed bottleneck architectures fail to adapt to the heterogeneous information densities inherent in diverse spoofing artifacts. Moreover, standard augmentation often leads to a pathological robustness-accuracy trade-off. In this paper, we propose a unified ADD framework to address these challenges. Our approach integrates: (1) an Adaptive Audio Feature Integration (AAFI) module that dynamically selects optimal latent dimensions via a compression pool and a stochastic exploration mechanism, ensuring flexible representations across multiple types of dynamic attacks; and (2) a Mel-Band Adversarial Perturbation (Mel-BAP) strategy that applies targeted regularization in the mel-spectrogram domain to encourage the model to learn invariant decision boundaries. Extensive evaluations on benchmarks including ADD 2022, FakeOrReal, and In-the-Wild demonstrate that our model based on Whisper-small achieves a state-of-the-art average Equal Error Rate (EER) of 1.51%. Compared to existing SOTA models such as the 3-billion-parameter Resemble-Detect-3B-Omni, our framework reduces the parameter count by up to 91.6% while achieving a 60.7% improvement in average EER. In a separate cross-lingual evaluation, the Whisper-base configuration achieves EERs of 3.91-5.65% on five unseen languages. The framework also maintains strong resilience to real-world acoustic distortions. This work provides a scalable and efficient plug-in solution for enhancing trust and security in the audio Web ecosystem.

Key findings

The proposed framework achieves a state-of-the-art average Equal Error Rate (EER) of 1.51% on key benchmarks, outperforming larger SOTA models by a significant margin with up to 91.6% fewer parameters. It demonstrates exceptional zero-shot transferability to unseen languages (EERs of 3.91-5.65% on five languages) and maintains strong resilience against real-world acoustic distortions and adaptive adversarial attacks. The adaptive feature integration and targeted adversarial perturbation effectively address challenges of generalization and robustness in audio deepfake detection.

Approach

The proposed framework integrates two main components: (1) an Adaptive Audio Feature Integration (AAFI) module which uses a compression pool and stochastic exploration to dynamically select optimal latent dimensions for audio features, and (2) a Mel-Band Adversarial Perturbation (Mel-BAP) strategy that applies targeted adversarial regularization in the mel-spectrogram domain. This combination aims to provide flexible representations and learn invariant decision boundaries, building upon a frozen Whisper encoder and a trainable Whisper decoder.

Datasets

ADD 2022, FakeOrReal, FakeOrReal-norm, In-the-Wild, ASVspoofing 2019, ASVspoofing 2021 LA, SpeechFake (Bilingual Dataset, MD), CodecFake.

Model(s)

Whisper-base, Whisper-small (backbones), AAFI module, Mel-BAP strategy.

Author countries

China, United Kingdom

← Previous