Time-Frequency Consistency Learning for Robust Speech Deepfake Detection

View on arXiv ← Back to list

Authors: Jun Xue, Zhuolin Yi, Yanzhen Ren, Yihuan Huang, Jiayu Xiong, Yi Chai, Guanxiang Feng, Jiajun Liu, Tong Zhang

Published: 2026-07-20 09:51:03+00:00

Comment: Accepted by ACM MM 2026

AI Summary

This research investigates the robustness of speech deepfake detection (SDD) models under real-world acoustic front-end (AFE) processing pipelines, finding significant performance degradation. To address this, the authors propose a Time-Frequency Consistency Learning (TFCL) framework that learns invariant spoofing representations by employing attention-driven soft alignment and frequency-domain structural consistency constraints.

Abstract

Recently, speech deepfake detection (SDD) has achieved significant progress. However, its robustness evaluation remains largely confined to controlled additive noise scenarios, lacking systematic investigation of the complex distortions introduced by acoustic front-end (AFE) processing pipelines in real-world deployments. In this work, we simulate a unified AFE pipeline comprising acoustic echo cancellation, noise suppression, automatic gain control, and voice activity detection (VAD), and conduct a comprehensive evaluation of current state-of-the-art models. The results show that the nonlinear and time-frequency coupled distortions introduced by AFE significantly degrade detection performance. To address this issue, we propose a Time-Frequency Consistency Learning (TFCL) framework, which aims to learn invariant spoofing representations that remain stable before and after AFE processing. We observe that AFE not only introduces temporal misalignment (e.g., segment-level shifts caused by VAD), but also weakens or distorts critical frequency-domain cues. To this end, TFCL employs an attention-driven soft alignment mechanism to capture cross-temporal dependencies, along with frequency-domain structural consistency constraints to enforce feature invariance. As a result, the model is able to maintain stable representations under both temporal perturbations and spectral distortions. Extensive experimental results demonstrate that the proposed method effectively mitigates the performance degradation caused by AFE processing, significantly improving the robustness of SDD in real-world scenarios. The code is available at https://github.com/JunXue-tech/TFCL.

Key findings

State-of-the-art SDD models suffer severe performance degradation when exposed to complex, cascaded distortions from acoustic front-end processing. The TFCL framework significantly mitigates this degradation, achieving more stable performance across various AFE conditions and different model backbones. This robustness is attributed to learning invariant spoofing representations by addressing both temporal dependency disruptions and frequency structural distortions.

Approach

The proposed TFCL framework uses a dual-branch consistency learning approach, employing a temporal dependency consistency module with a bidirectional cross-attention mechanism for temporal alignment and a frequency-domain structural consistency module to preserve stable spectral relationships. These are jointly optimized with the classification loss.

Datasets

ASVspoof2019 LA, MUSAN, DNS

Model(s)

XLSR+AASIST, XLSR+MultiConv, XLSR+Nes2Net (as baselines, TFCL integrated)

Author countries

China

← Previous