Audio Deepfake Detection with Half-Truth Localisation Using Cross-Attentive Feature Fusion

Authors: S. Sutharya, Remya K. Sasi

Published: 2026-05-28 07:47:22+00:00

Comment: 13 pages, 5 figures, 11 tables

AI Summary

This paper introduces CAFNet, a novel architecture for audio deepfake detection that jointly performs ternary classification (real, fully-fake, or half-truth) and temporal localization of manipulated segments in a single forward pass. CAFNet leverages cross-attentive feature fusion of MFCC, LFCC, and Chroma-STFT features. It establishes the first localization baseline on the MLADDC T3 benchmark and significantly outperforms larger pre-trained models on binary detection with far fewer parameters.

Abstract

Audio deepfake detection is well-studied as a binary problem, but partially manipulated speech, where a short synthesised segment is spliced into an otherwise genuine utterance, poses a harder and more realistic threat. Detecting such half-truth audio requires not only distinguishing it from real and fully fake speech, but also localising where the manipulation occurs. We present CAFNet, a 576k-parameter architecture that addresses both tasks jointly: it performs ternary classification (real, fully-fake, or half-truth) and regresses the temporal boundaries of the synthesised region in a single forward pass. CAFNet fuses Mel-Frequency Cepstral Coefficient (MFCC), Linear-Frequency Cepstral Coefficient (LFCC), and Chroma Short-Time Fourier Transform (Chroma-STFT) features through parallel depthwise-separable convolution branches with cross-attention, followed by a Bidirectional Long Short-Term Memory (BiLSTM) regression head for boundary prediction. On the combined Multi-Lingual Audio Deepfake Detection Corpus (MLADDC) T2+T3 test set, CAFNet achieves 92.71% accuracy and macro Area Under the Curve (AUC) of 0.9910, with boundary localisation Mean Absolute Error (MAE) of 0.075s and a median error of 0.052s. On binary detection, it achieves 96.76% accuracy and 3.20% Equal Error Rate (EER), outperforming fine-tuned XLS-R 300M (78.31%) and AST 87M (93.03%) at over 500 times fewer parameters. A cross-dataset study further shows that standard fine-tuning collapses cross-domain representations even under reduced backbone learning rates.

Key findings

CAFNet achieved 92.71% accuracy and macro AUC of 0.9910 for ternary classification on MLADDC T2+T3, with a mean absolute error of 0.075s for boundary localization. It outperformed fine-tuned XLS-R 300M and AST 87M in binary detection accuracy with over 500 times fewer parameters. The study also demonstrated that standard fine-tuning paradigms lead to a collapse of cross-domain representations, highlighting a significant challenge for generalization in audio deepfake detection.

Approach

CAFNet processes Mel-Frequency Cepstral Coefficient (MFCC), Linear-Frequency Cepstral Coefficient (LFCC), and Chroma Short-Time Fourier Transform (Chroma-STFT) features through parallel depthwise-separable convolution branches. These feature streams are then fused using cross-attention, followed by a Bidirectional Long Short-Term Memory (BiLSTM) regression head for temporal boundary prediction and a dense head for ternary classification.

Datasets

Multi-Lingual Audio Deepfake Detection Corpus (MLADDC) T2+T3, FoR-norm, WaveFake, ASVspoof 2019 LA, In-the-Wild

Model(s)

CAFNet (custom architecture with depthwise-separable convolutions, cross-attention, BiLSTM), MFAAN, XLS-R 300M, AST 87M

Author countries

India

← Previous