Disentangled Global-Local Feature Learning with E-Branchformer for Audio Deepfake Detection

View on arXiv ← Back to list

Authors: Phuong Tuan Dat, Ho Bao Thu, Nguyen Tran Trung, Pham Viet Hoang, Nguyen Thi Thu Trang

Published: 2026-09-08 16:06:42+00:00

Comment: Accepted to SALMA Workshop @ EMNLP 2026

AI Summary

This paper introduces a novel E-Branchformer-based architecture for audio deepfake detection, leveraging self-supervised speech representations. The model employs parallel branches to simultaneously capture global contextual dependencies and local temporal patterns, achieving state-of-the-art performance on ASVspoof 2021 LA, DF, and In-the-Wild datasets.

Abstract

The rapid advancement of voice synthesis technologies such as text-to-speech and voice conversion poses significant threats to speech-based authentication systems, necessitating robust deepfake detection methods. In this work, we propose a novel E-Branchformer-based architecture that effectively leverages self-supervised speech representations for audio deepfake detection. Our model employs parallel branches to simultaneously capture global contextual dependencies through multi-head self-attention and local temporal patterns through convolutional processing. To enhance discriminative capability, we integrate depthwise convolution and Squeeze-and-Excitation modules that enrich the classification token with refined patch token information after feature merging. Extensive experiments on ASVspoof 2021 LA, DF, and In-the-Wild datasets demonstrate state-of-the-art performance with equal error rates of 0.88%, 1.85%, and 6.30% respectively, substantially outperforming existing methods. Comprehensive ablation studies validate that the dual-branch architecture provides complementary discriminative information, Squeeze-and-Excitation Aggregation significantly improves SSL feature integration, and the combination of DWConv and SE modules is critical for effective class token enhancement. The superior performance on real-world scenarios demonstrates strong generalization capability to diverse acoustic conditions and unseen spoofing attacks.

Key findings
The proposed E-Branchformer model achieves state-of-the-art performance with EERs of 0.88% on ASVspoof 2021 LA, 1.85% on ASVspoof 2021 DF, and 6.30% on the In-the-Wild dataset. Ablation studies confirm that the dual-branch architecture, Squeeze-and-Excitation Aggregation, and the combination of DWConv and SE modules are critical for enhancing discriminative capability and achieving superior generalization to real-world scenarios.
Approach
The proposed model utilizes an E-Branchformer architecture with parallel branches: one for global contextual dependencies using multi-head self-attention and another for local temporal patterns via convolutional processing. It integrates depthwise convolution and Squeeze-and-Excitation modules to enrich the classification token with refined patch token information after feature merging, using self-supervised speech representations (XLS-R) as the front-end.
Datasets
ASVspoof 2019 LA, ASVspoof 2021 LA, ASVspoof 2021 DF, ASVspoof 5, In-the-Wild (ITW)
Model(s)
E-Branchformer, XLS-R 300M (front-end), Conformer (baseline), XLSR-Conformer, XLSR-Conformer+TCM, XLSR-AASIST, WavLM-MFA, XLSR-SLS, XLSR-MoE, XLSR-Mamba, Nes2Net, Nes2Net-X
Author countries
Singapore, Vietnam
← Previous
Next →