MixFake: Benchmarking and Enhancing Audio Deepfake Detection in Diverse Real-world Mixed Audio

Authors: Qingcao Li, Yipeng Lin, Weichen Lian, Zhongjie Ba, Peng Cheng, Zhichao Lian

Published: 2026-05-22 03:33:36+00:00

Comment: Accepted by ICME2026

AI Summary

This paper introduces MixFake, a large-scale benchmark dataset for audio deepfake detection in diverse real-world mixed audio environments, where traditional semantic-centric self-supervised learning (SSL) models often fail. To overcome this, they propose a Multi-stream Prompt Tuning framework that injects signal-level priors, specifically frequency and texture information via Hilbert-Huang Transform and Teager-Kaiser Energy Operator, into SSL backbones. This approach significantly enhances detection performance in both foreground and complex background mixed-audio scenarios.

Abstract

Speech deepfake detection has achieved remarkable success in clean environments but faces significant challenges in complex, real-world scenarios where speech is often mixed with background music or noise. Current state-of-the-art methods rely on semantic features from self-supervised learning (SSL) models, which often fail when processing non-speech or mixed-source audio. In this paper, we first introduce MixFake, a large-scale benchmark dataset designed to simulate diverse acoustic environments with varying SNR levels and mixed authenticity components. To address the semantic-centric limitation, we propose a Multi-stream Prompt Tuning framework that injects signal-level priors into SSL backbones. By integrating base, frequency, and texture streams through deep prompt injection, our model effectively captures acoustic artifacts. Experimental results demonstrate that our method significantly outperforms existing baselines, achieving a 0.95% EER in foreground detection and a substantial 7.72% absolute improvement in complex background detection tasks. Our dataset and code are available at https://github.com/saltfish233/MixFake.

Key findings

The proposed method achieved state-of-the-art performance, with a 0.95% EER in foreground speech detection. It demonstrated a substantial 7.72% absolute improvement in the more challenging complex background detection task. The approach also showed superior robustness across diverse SNR levels and strong generalization capabilities in cross-dataset evaluations.

Approach

The proposed Multi-stream Prompt Tuning framework extends an SSL backbone (XLSR-AASIST) by injecting signal-level priors through deep prompt tuning across three streams. It utilizes a Base Stream for foundational learnable prompts, a Frequency Stream leveraging Hilbert-Huang Transform (HHT) for instantaneous frequency anomaly detection, and a Texture Stream employing Teager-Kaiser Energy Operator (TKEO) for non-linear energy fluctuations. These multi-dimensional priors are injected layer-wise into Transformer blocks to capture acoustic artifacts in mixed-source audio.

Datasets

MixFake, ASVspoof 2019 LA, In-the-Wild

Model(s)

XLSR-AASIST (based on XLS-R and Graph Attention Network), Multi-stream Prompt Tuning framework (integrating Hilbert-Huang Transform (HHT) and Teager-Kaiser Energy Operator (TKEO))

Author countries

China

← Previous