CRAF: Cross-View Residual-Aware Fusion for Deepfake Speech Detection

Authors: Minh-Xuan Phan, Khalid Zaman, Candy Olivia Mawalim, Masashi Unoki

Published: 2026-09-12 09:56:45+00:00

AI Summary

This paper introduces CRAF, a cross-view residual-aware fusion framework for deepfake speech detection that leverages complementary representations from Self-Supervised Learning (SSL) models and Auditory Large Language Models (ALLMs). CRAF addresses the challenge of generalizing to unseen spoofing attacks by disentangling shared and view-specific information, using ALLM-guided attention to enrich SSL representations and extracting complementary SSL residual information. Experiments on ASVspoof 5 demonstrate CRAF's robustness and improved performance against various unseen spoofing attacks.

Abstract

Recent advances in speech synthesis and voice conversion have made deepfake speech increasingly realistic, making generalization to unseen spoofing attacks a critical challenge. Pretrained speech and audio models offer a promising direction for improving robustness to such unseen attacks. Self-supervised learning (SSL) models capture fine-grained, low-level acoustic characteristics, whereas Auditory Large Language Models (ALLMs) provide higher-level contextual representations. These complementary views can provide useful cues for improving generalization to unseen attacks. However, direct fusion does not explicitly disentangle information shared across the two views from view-specific complementary information, limiting effective cross-view integration. To address this, we propose CRAF, a cross-view residual-aware fusion framework that uses ALLM-guided cross-view attention to enrich SSL representations and adopts ALLM as a high-level reference to separate ALLM-explainable information from complementary SSL residual information. The residual is selectively refined through adaptive gating and integrated through SSL-primary fusion. Experiments on ASVspoof 5 show that CRAF with Kimi-Audio achieves an EER of 5.96% and a minDCF of 0.1192, demonstrating robustness to unseen spoofing attacks.

Key findings

CRAF consistently improved deepfake speech detection performance over single-encoder baselines on the ASVspoof 5 evaluation set, demonstrating enhanced robustness to unseen spoofing attacks. CRAF with Kimi-Audio achieved the best performance with an EER of 5.96% and a minDCF of 0.1192. Ablation studies confirmed that all CRAF components (cross-view attention, residual learning, and SSL-primary fusion) contribute to its effectiveness, providing complementary benefits rather than redundant functionality.

Approach

CRAF combines SSL and ALLM representations in an asymmetric design, using ALLM as a high-level guide to enrich SSL features via cross-view attention. It then identifies and refines a 'residual' of SSL-specific information not covered by ALLM, which is adaptively fused with the ALLM-guided SSL representation and the original SSL stream. This fusion strategy aims to preserve fine-grained acoustic details while incorporating high-level contextual information.

Datasets

ASVspoof 5

Model(s)

XLS-R 300M (SSL encoder), Step-Audio, Qwen-Audio, Kimi-Audio (ALLM encoders), AASIST (backend classifier)

Author countries

Japan

← Previous