Interpreting Multi-Branch Anti-Spoofing Architectures: Correlating Internal Strategy with Empirical Performance

Authors: Ivan Viakhirev, Kirill Borodin, Mikhail Gorodnichev, Grach Mkrtchian

Published: 2026-02-14 20:15:54+00:00

Comment: Published at MDPI Mathematics (see at https://www.mdpi.com/2227-7390/14/2/381)

Journal Ref: Mathematics 14 (2026)

AI Summary

This paper introduces a framework for interpreting the internal decision dynamics of multi-branch audio anti-spoofing networks like AASIST3, moving beyond traditional input-level saliency. It models intermediate activations with covariance operators to generate spectral signatures, which then train a CatBoost meta-classifier to derive TreeSHAP-based branch attributions and confidence scores. The analysis identifies four operational archetypes and exposes a critical 'Flawed Specialization' mode where the model confidently makes incorrect predictions for certain spoofing attacks.

Abstract

Multi-branch deep neural networks like AASIST3 achieve state-of-the-art comparable performance in audio anti-spoofing, yet their internal decision dynamics remain opaque compared to traditional input-level saliency methods. While existing interpretability efforts largely focus on visualizing input artifacts, the way individual architectural branches cooperate or compete under different spoofing attacks is not well characterized. This paper develops a framework for interpreting AASIST3 at the component level. Intermediate activations from fourteen branches and global attention modules are modeled with covariance operators whose leading eigenvalues form low-dimensional spectral signatures. These signatures train a CatBoost meta-classifier to generate TreeSHAP-based branch attributions, which we convert into normalized contribution shares and confidence scores (Cb) to quantify the model's operational strategy. By analyzing 13 spoofing attacks from the ASVspoof 2019 benchmark, we identify four operational archetypes-ranging from Effective Specialization (e.g., A09, Equal Error Rate (EER) 0.04%, C=1.56) to Ineffective Consensus (e.g., A08, EER 3.14%, C=0.33). Crucially, our analysis exposes a Flawed Specialization mode where the model places high confidence in an incorrect branch, leading to severe performance degradation for attacks A17 and A18 (EER 14.26% and 28.63%, respectively). These quantitative findings link internal architectural strategy directly to empirical reliability, highlighting specific structural dependencies that standard performance metrics overlook.

Key findings

The study identified four operational archetypes: Effective Specialization, Effective Consensus, Ineffective Consensus, and Flawed Specialization, dynamically adopted by AASIST3 for different attacks. A crucial finding is the 'Flawed Specialization' mode (e.g., A17, A18 with EERs of 14.26% and 28.63%), where the model exhibits high confidence in an incorrect branch, leading to severe performance degradation. This analysis quantitatively links internal architectural strategy directly to empirical reliability and exposes structural dependencies, such as single points of failure in successful specialization scenarios, that standard performance metrics often overlook.

Approach

The authors model intermediate activations from 14 branches and global attention modules of the AASIST3 architecture using covariance operators to create low-dimensional spectral signatures. These signatures are then fed into a CatBoost meta-classifier, which generates TreeSHAP-based branch attributions. These attributions are converted into normalized contribution shares and confidence scores to quantify the model's operational strategy and correlate it with empirical performance.

Datasets

ASVspoof 2019 benchmark, ASVspoof 2019 Logical Access (LA) dataset

Model(s)

AASIST3 architecture, RawNet2-based encoder (as part of AASIST3), CatBoost classifier

Author countries

Russia

← Previous