Joint Optimization of Speaker and Spoof Detectors for Spoofing-Robust Automatic Speaker Verification

Authors: Oğuzhan Kurnaz, Jagabandhu Mishra, Tomi H. Kinnunen, Cemal Hanilçi

Published: 2025-10-02 09:04:31+00:00

AI Summary

This paper proposes a modular yet jointly optimized architecture for spoofing-robust automatic speaker verification (SASV), integrating outputs from speaker and spoof detectors via trainable back-end classifiers. The approach directly optimizes the back-end using the architecture-agnostic detection cost function (a-DCF) as a training objective. Experiments demonstrate that nonlinear score fusion and a combination of weighted cosine scoring for speaker detection with SSL-AASIST for spoof detection achieve state-of-the-art performance.

Abstract

Spoofing-robust speaker verification (SASV) combines the tasks of speaker and spoof detection to authenticate speakers under adversarial settings. Many SASV systems rely on fusion of speaker and spoof cues at embedding, score or decision levels, based on independently trained subsystems. In this study, we respect similar modularity of the two subsystems, by integrating their outputs using trainable back-end classifiers. In particular, we explore various approaches for directly optimizing the back-end for the recently-proposed SASV performance metric (a-DCF) as a training objective. Our experiments on the ASVspoof 5 dataset demonstrate two important findings: (i) nonlinear score fusion consistently improves a-DCF over linear fusion, and (ii) the combination of weighted cosine scoring for speaker detection with SSL-AASIST for spoof detection achieves state-of-the-art performance, reducing min a-DCF to 0.196 and SPF-EER to 7.6%. These contributions highlight the importance of modular design, calibrated integration, and task-aligned optimization for advancing robust and interpretable SASV systems.

Key findings

Nonlinear score fusion consistently improves the a-DCF metric compared to linear fusion. The proposed system, combining weighted cosine scoring for ASV and SSL-AASIST for CM, achieves state-of-the-art performance with a min a-DCF of 0.196 and SPF-EER of 7.6%. This highlights the importance of modular design, calibrated integration, and task-aligned optimization, demonstrating enhanced robustness against diverse spoofing attacks, including adversarial ones.

Approach

The authors propose a modular SASV system that integrates pre-trained ASV and CM embedding extractors. Their outputs are combined using trainable back-end classifiers, specifically a nonlinear score fusion module operating on calibrated ASV and CM scores. The system is jointly optimized end-to-end for the a-DCF metric, complemented by auxiliary binary cross-entropy losses, and employs weighted cosine scoring for the ASV branch.

Datasets

ASVspoof 5 dataset

Model(s)

ECAPA-TDNN, WavLM-TDNN, ReDimNet (for ASV); AASIST, SSL-AASIST (for CM)

Author countries

Turkey, Finland

← Previous