From Scores to Evidence: Auditable Decisions Can Improve Speech Deepfake Detection

Authors: Mengzhe Geng, Yujia Lu, Patrick Littell, Manuela Kunz, Xie Chen

Published: 2026-09-08 15:38:57+00:00

AI Summary

This paper introduces an auditable decision record for speech deepfake detection that preserves four aligned cues (passive score, keyed-probe score, retrieval support, and speaker-profile margin) into a late calibration step. This approach allows for scalar decisions while retaining the underlying evidence for review and analysis, leading to improved detection performance and better error coverage at fixed review budgets. The main contribution is not a standalone stronger detector, but rather a method to transparently present the evidence behind each decision.

Abstract

Speech deepfakes can mimic a speaker's voice convincingly enough to deceive listeners and automated systems. This has driven strong progress in speech deepfake detection, but most detectors still end with one score per utterance. That score is useful for ranking systems, yet it says little about why a borderline item should be trusted, deferred, or reviewed. Two utterances can fall in the same score band for different reasons, for example because passive and retrieval evidence disagree or because the keyed probe is unavailable. We ask whether the final decision can remain scalar without discarding that provenance. We answer this question with an auditable decision record that carries four aligned cues into a late calibration step: a passive detector score, a conditional keyed-probe score on a marked derivative, retrieval support, and a speaker-profile margin, together with explicit disagreement coordinates. On the 4,080-example ASVspoof 5 Track 1 matched subset, the fixed retrieval-augmented rule improves on retrieval-only evidence, from 15.84 percent to 11.91 percent EER, and late calibration over the full record reaches 8.43 percent EER. At a 33.75 percent review budget, the exposed cue union covers 82.85 percent of the calibrated model's errors. The best passive WavLM run still reaches 6.71 percent EER, so we do not present the decision record as a stronger standalone detector. Its contribution is to preserve the evidence behind each surfaced utterance while still producing one operating score for thresholding and review.

Key findings

The proposed auditable decision record with late calibration achieved an 8.43% EER on the ASVspoof 5 matched subset, outperforming retrieval-only evidence (15.84% EER) and a fixed retrieval-augmented rule (11.91% EER). At a 33.75% review budget, the exposed cue union covered 82.85% of the calibrated model's errors, demonstrating its utility in improving review processes even if it doesn't surpass the best passive WavLM baseline (6.71% EER) as a standalone detector.

Approach

The authors develop an auditable decision record that compiles four distinct cues for each utterance: a passive detector score, a conditional keyed-probe score, retrieval support, and a speaker-profile margin. These cues, along with explicit disagreement coordinates, are then fed into a late calibration step to produce a single operating score, while keeping the original evidence visible for review.

Datasets

ASVspoof 5 Track 1 development set (matched subset of 4,080 examples), In-The-Wild, WaveFake, ASVspoof 2021

Model(s)

WavLM-large, RawNet2, HuBERT-large, Wav2Vec2-large, spectrogram-CNN

Author countries

Canada, Hong Kong, China

← Previous