Joint Optimization of Speaker and Spoof Detectors for Spoofing-Robust Automatic Speaker Verification

View on arXiv ← Back to list

Authors: Oğuzhan Kurnaz, Jagabandhu Mishra, Tomi H. Kinnunen, Cemal Hanilçi

Published: 2025-10-02 09:04:31+00:00

AI Summary

This study proposes a modular, yet jointly optimized architecture for Spoofing-Robust Automatic Speaker Verification (SASV) by integrating separately trained ASV and CM subsystems. The system uses trainable back-end classifiers and non-linear score fusion, exploring direct optimization of the fusion back-end using the architecture-agnostic detection cost function (a-DCF) as the training objective. This approach highlights the importance of modular design, calibrated integration, and task-aligned optimization for robust SASV systems.

Abstract

Spoofing-robust speaker verification (SASV) combines the tasks of speaker and spoof detection to authenticate speakers under adversarial settings. Many SASV systems rely on fusion of speaker and spoof cues at embedding, score or decision levels, based on independently trained subsystems. In this study, we respect similar modularity of the two subsystems, by integrating their outputs using trainable back-end classifiers. In particular, we explore various approaches for directly optimizing the back-end for the recently-proposed SASV performance metric (a-DCF) as a training objective. Our experiments on the ASVspoof 5 dataset demonstrate two important findings: (i) nonlinear score fusion consistently improves a-DCF over linear fusion, and (ii) the combination of weighted cosine scoring for speaker detection with SSL-AASIST for spoof detection achieves state-of-the-art performance, reducing min a-DCF to 0.196 and SPF-EER to 7.6%. These contributions highlight the importance of modular design, calibrated integration, and task-aligned optimization for advancing robust and interpretable SASV systems.

Key findings

Nonlinear score fusion consistently outperformed linear fusion across all experimental settings in terms of a-DCF performance. The optimal combination of ReDimNet and SSL-AASIST with weighted cosine scoring achieved state-of-the-art performance on the ASVspoof 5 evaluation set, resulting in a minimum a-DCF of 0.196 and SPF-EER of 7.6%. Using self-supervised CM models like SSL-AASIST significantly improved spoofing robustness compared to vanilla AASIST.

Approach

The system fuses calibrated log-likelihood ratio (LLR) scores derived from separate ASV and CM subsystems using a trainable non-linear fusion function. The ASV branch utilizes weighted cosine similarity on embeddings, while the CM branch employs an MLP classifier on fused ASV/CM embeddings. The trainable components are jointly optimized end-to-end using a combination of the a-DCF loss and auxiliary BCE losses.

Datasets

ASVspoof 5 (Track 2: SASV).

Model(s)

ECAPA-TDNN, WavLM-TDNN, ReDimNet (ASV backbones), AASIST, SSL-AASIST (CM backbones).

Author countries

Turkey, Finland

← Previous