Tandem Assessment of Spoofing Countermeasures and Automatic Speaker Verification: Fundamentals

View on arXiv ← Back to list

Authors: Tomi Kinnunen, Héctor Delgado, Nicholas Evans, Kong Aik Lee, Ville Vestman, Andreas Nautsch, Massimiliano Todisco, Xin Wang, Md Sahidullah, Junichi Yamagishi, Douglas A. Reynolds

Published: 2020-07-12 12:44:08+00:00

Comment: Published in IEEE/ACM Transactions on Audio, Speech, and Language Processing (doi updated)

AI Summary

This paper extends the tandem detection cost function (t-DCF) as a risk-based metric for assessing spoofing countermeasures (CMs) when deployed in tandem with automatic speaker verification (ASV) systems. It presents a simplified version of the t-DCF, analyzes a special case for a fixed ASV system, and provides new insights and empirical analyses using the ASVspoof 2019 database. The work aims to foster closer collaboration between anti-spoofing and ASV research communities by promoting a more application-relevant assessment approach than traditional Equal Error Rate (EER).

Abstract

Recent years have seen growing efforts to develop spoofing countermeasures (CMs) to protect automatic speaker verification (ASV) systems from being deceived by manipulated or artificial inputs. The reliability of spoofing CMs is typically gauged using the equal error rate (EER) metric. The primitive EER fails to reflect application requirements and the impact of spoofing and CMs upon ASV and its use as a primary metric in traditional ASV research has long been abandoned in favour of risk-based approaches to assessment. This paper presents several new extensions to the tandem detection cost function (t-DCF), a recent risk-based approach to assess the reliability of spoofing CMs deployed in tandem with an ASV system. Extensions include a simplified version of the t-DCF with fewer parameters, an analysis of a special case for a fixed ASV system, simulations which give original insights into its interpretation and new analyses using the ASVspoof 2019 database. It is hoped that adoption of the t-DCF for the CM assessment will help to foster closer collaboration between the anti-spoofing and ASV research communities.

Key findings

The study demonstrates that spoofing countermeasures significantly reduce the tandem system cost compared to no CMs, though none of the evaluated CMs reached the theoretical ASV floor. The ASV-constrained t-DCF reveals that CM performance rankings and optimal thresholds can vary based on the assumed spoofing prior. Furthermore, using t-DCF for empirical threshold selection can lead to substantially lower costs compared to EER-based thresholds, emphasizing the need for application-directed metrics.

Approach

The authors extend the tandem detection cost function (t-DCF) to evaluate spoofing countermeasures (CMs) and Automatic Speaker Verification (ASV) systems. They propose a simplified t-DCF with fewer parameters and an ASV-constrained variant for a fixed ASV system. The approach involves analyzing the impact of different detection costs and prior probabilities of target, non-target, and spoof trials on the overall system performance, guiding optimal threshold selection.

Datasets

ASVspoof 2015, ASVspoof 2017, ASVspoof 2019 (Logical Access - LA), ASVspoof 2019 (Physical Access - PA), VoxCeleb1, VoxCeleb2

Model(s)

TDNN-based x-vector speaker embeddings with a PLDA backend (for ASV system), various CM systems submitted to the ASVspoof challenges (for evaluation by t-DCF)

Author countries

Finland, Spain, France, Japan, USA

← Previous