What Do Deepfake Speech Detectors Actually Hear?

Authors: Vojtěch Staněk, Veronika Jirmusová, Anton Firc, Kamil Malinka, Jakub Reš, Martin Perešíni

Published: 2026-06-09 14:21:45+00:00

Comment: Accepted to Interspeech 2026

AI Summary

This paper introduces an audio-native explainability pipeline using Integrated Gradients on time-aligned self-supervised representations to analyze the decision-making of deepfake speech detectors. By applying this method to WavLM-based AASIST, CA-MHFA, and SLS detectors on ASVspoof 5, the authors uncover that these detectors, despite similar performance, rely on distinct cues such as non-speech regions, localized phoneme artifacts, and spectral integrity. The findings are validated through causal masking experiments, demonstrating which cues drive each detector's decisions.

Abstract

Deepfake speech detectors often output a single score without explaining why an audio sample is flagged, where in the signal the evidence lies, or what cues drive the decision. We propose an audio-native explainability pipeline using Integrated Gradients on time-aligned self-supervised representations to localize decision evidence over time. We apply the proposed method to three WavLM-based detectors (AASIST, CA-MHFA, SLS) on ASVspoof 5 and manually annotate the highest-attribution regions to provide a semantic meaning of the most important cues. Despite similar performance, the detectors rely on different cues: AASIST emphasizes non-speech/environment cues, CA-MHFA focuses on localized phoneme artifacts, and SLS relies on word boundaries and spectral integrity. We move beyond speculative reasoning and validate our findings by causal masking of the primary detector cues. Observed performance degradation further supports the explained detector semantics.

Key findings

The study found that AASIST primarily acts as an environmental anomaly detector, focusing on non-speech regions, while CA-MHFA is a highly localized detector targeting specific articulation and phoneme artifacts. SLS, conversely, monitors overall spectral integrity and word boundaries. All three detectors exhibit a shared vulnerability to heavy audio compression, misinterpreting compression artifacts as synthetic generation cues.

Approach

The authors propose an audio-native explainability pipeline that leverages Integrated Gradients (IG) on time-aligned self-supervised (WavLM) representations to localize decision evidence in deepfake speech. They sum IG attributions across layers and features to create a temporal attribution map. This map, along with the audio signal and spectrogram, is used by human annotators to semantically label primary cue regions, and these findings are then causally validated through targeted masking experiments.

Datasets

ASVspoof 5

Model(s)

AASIST, CA-MHFA, SLS (all utilizing WavLM Base+ as front-end)

Author countries

Czech Republic

← Previous