Trajectory Dynamics in Self-Supervised Learning Latent Space for Audio Deepfake Detection

Authors: Tomás Andrade Weber

Published: 2026-08-13 23:06:34+00:00

Comment: 5 pages, 1 figure

AI Summary

This research proposes a novel audio deepfake detection method that exploits the physiological constraints of human speech, which manifest as structured trajectory dynamics in the latent space of Self-Supervised Learning (SSL) models. The approach trains a causal LSTM next-frame predictor on bonafide speech only, then uses its internal states with a supervised MLP, achieving competitive or state-of-the-art performance across multiple benchmarks, particularly for challenging cross-corpus deepfakes.

Abstract

Human speech production is constrained by physiology, giving rise to characteristic temporal structure on acoustic signals. We hypothesise that these constraints manifest as structured trajectory dynamics in the latent space of Self-Supervised Learning (SSL) models, and that synthetic speech violates them detectably. To test this hypothesis, we train a causal Long Short-Term Memory (LSTM) next-frame predictor on bonafide speech only (Stage 1), using the deepfake-specialised SSL backbone Wav2Vec2-Large-AntiDeepfake, and compare against a static global-average-pooling baseline using identical features, thus isolating the contribution of temporal modelling. A supervised Stage 2, which trains a Multi-Layer Perceptron on the frozen LSTM internal states using labelled data, is included to characterise the role of spoof supervision. Our system achieves competitive or state-of-the-art performance across six benchmarks: ASVspoof 2019/2021, Codecfake, In-the-Wild, MLAAD-EN, and Deepfake-Eval-2024, including best published EER on ASVspoof 2021 (0.75\\%) and, notably, Stage 1 trained on bonafide speech only surpasses the published supervised baseline from the same backbone on DE2024 (30.35\\%). On near-domain benchmarks, static and dynamic approaches perform comparably. On harder cross-corpus benchmarks with diverse synthesis methods, trajectory dynamics provide substantial gains, confirming that temporal physiological constraints carry detection signal beyond utterance-level statistics.

Key findings

Trajectory dynamics provide substantial gains in detecting deepfakes, especially on harder cross-corpus benchmarks like MLAAD-EN and DE2024, confirming that temporal physiological constraints carry detection signals beyond utterance-level statistics. The one-class Stage 1, trained solely on bonafide speech, often outperforms the supervised Stage 2 on challenging out-of-domain benchmarks, suggesting better generalization to unseen synthesis conditions.

Approach

The method involves a two-stage process. Stage 1 trains a causal LSTM next-frame predictor exclusively on bonafide speech, using features from the Wav2Vec2-Large-AntiDeepfake SSL backbone, to learn the natural trajectory dynamics. Stage 2 then uses the frozen LSTM's internal states to train a Multi-Layer Perceptron with labeled data (bonafide and spoof) to classify deepfakes, isolating the contribution of temporal modeling.

Datasets

ASVspoof 2019 train (for training), ASVspoof 2019 eval, ASVspoof 2021 eval, Codecfake, In-the-Wild, MLAAD-EN, Deepfake-Eval-2024 (DE2024).

Model(s)

Wav2Vec2-Large-AntiDeepfake (SSL backbone), Long Short-Term Memory (LSTM), Multi-Layer Perceptron (MLP).

Author countries

Spain

← Previous