CLRM: Cross-Layer Relation Modeling for Audio Deepfake Detection

Authors: Yangyang Qu, Siqing Qin, Kong Aik Lee, Massimiliano Todisco

Published: 2026-11-03 22:04:06+00:00

AI Summary

This paper introduces Cross-Layer Relation Modeling (CLRM) for audio deepfake detection, a method that explicitly encodes and fuses cross-layer relations from self-supervised learning (SSL) speech encoder representations alongside layer-specific features. CLRM reuses existing detectors' SSL encoders, identifies informative layer pairs based on speaker-relative representation geometry and detection performance, and fuses the resulting relation score with the original detector's score to enhance performance. The approach consistently reduces the mean equal error rate across various detectors and external datasets.

Abstract

Audio deepfake detection (ADD) aims to distinguish bonafide speech from synthetic or manipulated speech. Representations from different layers of self-supervised learning (SSL) speech encoders vary in their ability to distinguish bonafide from spoofed speech. Comparing representations of the same utterance across encoder depths may reveal useful spoofing cues, yet common layer-selection and aggregation methods do not provide these comparisons as detection inputs. We propose Cross-Layer Relation Modeling (\\ours{}), which explicitly encodes cross-layer relations alongside layer-specific features, making both directly available for ADD. CLRM reuses the existing detector's SSL encoder and identifies informative layer pairs by combining speaker-relative representation geometry with source-domain detection performance. The resulting relation score is fused with the detector's original score, requiring only the training of compact relation heads and fusion parameters while keeping the existing detector frozen. Across five external datasets, CLRM reduces the mean equal error rate (EER) for all three evaluated detectors, achieving relative reductions of 7.8--16.7%. Controlled comparisons with identical layer summaries and approximately matched auxiliary-head parameter counts also show a lower mean EER for explicit relations than direct concatenation, both before and after score fusion. These findings support explicit cross-layer comparisons as useful detection inputs for SSL-based audio deepfake detectors.


Key findings
CLRM significantly reduces the mean equal error rate (EER) for all three evaluated XLS-R-based detectors across five external datasets, achieving relative reductions of 7.8% to 16.7%. Explicit cross-layer relations provide more effective detection inputs than direct concatenation of layer summaries, both before and after score fusion. The approach demonstrates consistent improvements across stricter source protocols and initializations, highlighting its robustness and the value of explicit cross-layer comparisons for audio deepfake detection.
Approach
CLRM augments frozen SSL-based detectors by constructing a joint representation from two selected encoder layers, capturing individual features as well as pairwise discrepancies, interactions, and similarities. Compact relation heads learn an auxiliary detection score from this representation, which is then combined with the original detector's score through score-level fusion. Layer pairs are chosen by prioritizing those separated in depth but exhibiting similar speaker-relative organization on bonafide speech, with final selection based on detection performance.
Datasets
ASVspoof 2019 Logical Access (LA), ASVspoof 2021 LA, ASVspoof 2021 Deepfake (DF), ASVspoof 5 Eval, Fake-or-Real, In-the-Wild
Model(s)
XLS-R-300M encoder, XLSR-SLS, XLSR-Conformer-TCM, XLSR-AASIST
Author countries
France, China