Interpreting SSL Representations for Spoof Detection: a WavLM Study

Authors: Mohamed Mallat, Michele Panariello, Massimiliano Todisco, Nicholas Evans, Anthony Larcher

Published: 2026-10-16 14:26:55+00:00

Comment: 5 pages, 4 figures, 1 table. Published at IWBF 2026. An extended abstract of this work was presented as a poster at Odyssey 2026

Journal Ref: 2026 International Workshop on Biometrics and Forensics (IWBF)

AI Summary

This research investigates the interpretability of WavLM self-supervised learning (SSL) representations for audio deepfake detection, focusing on which layers encode spoof-relevant information and their acoustic properties. By employing linear probing, canonical correlation analysis (CCA), and layer subset search on the ASVspoof 5 database, the study reveals that mid-to-late layers are most discriminative, with voice quality features increasingly correlating with deeper layers.

Abstract

Self-supervised speech models such as WavLM achieve strong spoofing detection performance, yet which layers encode the most spoof-relevant information and what acoustic properties they capture remain opaque. We present a layer-wise interpretability study of WavLM using the ASVspoof 5 database, combining three complementary analyses: (i) per-layer linear probing with attack-level evaluation across speech synthesis, voice conversion, and adversarial attacks; (ii) a canonical correlation analysis (CCA) linking layer representations to spoofing-relevant acoustic features; (iii) an exhaustive grid search over layer subsets showing that use of five layers outperforms full thirteen-layer pooling. CCA analysis reveals that, while most acoustic features peak in alignment with mid-layers and then decay, voice quality measures jitter, shimmer, and the harmonics-to-noise ratio show the opposite trend, with correlation increasing toward later layers. Embedding visualisations reveal that deeper layers capture a continuous acoustic similarity among attacks rather than discrete categories, suggesting these layers encode fundamental synthesis artifacts rather than attack-specific signatures.

Key findings

Mid-to-late layers (L9-L11) of WavLM are the most discriminative for spoof detection, and a compact 5-layer subset (L7–9–10–11–12) can outperform full 13-layer pooling. Voice quality features (jitter, shimmer, HNR) show increasing correlation with deeper layers, while spectral and prosodic features peak in mid-layers, indicating a shift in what information is encoded. Deeper layers capture a continuous acoustic similarity among attacks, suggesting they encode fundamental synthesis artifacts rather than attack-specific signatures.

Approach

The authors analyze WavLM by (i) performing per-layer linear probing with attack-level evaluation, (ii) using Canonical Correlation Analysis (CCA) to link layer representations with spoofing-relevant acoustic features, and (iii) conducting an exhaustive grid search over layer subsets. This multi-faceted approach helps identify layer-specific discriminability and the acoustic properties captured at different depths.

Datasets

ASVspoof 5 database (Track 1 protocol)

Model(s)

WavLM Base model (as a frozen feature extractor)

Author countries

France

← Previous