Task-Aware Joint Pruning and Distillation for Efficient Audio Deepfake Detection

Authors: Miao He, Peng Cheng, Zhongjie Ba, Qing Wen, Li Lu, Xin Yang, Kui Ren

Published: 2026-10-04 14:34:27+00:00

Comment: 6 pages, 4 figures, accepted to Interspeech 2026

AI Summary

This paper introduces a Task-Aware Joint Pruning and Distillation framework to compress Self-Supervised Learning (SSL)-based audio deepfake detection models, which are typically computationally expensive. The framework combines cross-domain knowledge distillation with movement-guided structured pruning to preserve forgery-discriminative knowledge and critical model structures. This approach significantly reduces model parameters and FLOPs while maintaining competitive detection performance, enabling deployment on resource-constrained devices.

Abstract

Advances in speech synthesis have made deepfake speeches increasingly convincing, posing growing threats to security. While self-supervised learning (SSL) based detectors achieve state-of-the-art performance, their computational demands (typically 300M+ parameters) prevent deployment on resource-constrained devices. Existing compression methods, designed mainly for content-centric tasks, struggle to maintain competitive performance when directly adapted to deepfake detection. We propose a Task-Aware Joint Pruning and Distillation framework that combines cross-domain knowledge distillation with movement-guided structured pruning to transfer forgery-discriminative knowledge and preserve critical structures under aggressive compression. Our framework reduces the model to 31.9M parameters with 6.3$\\times$ FLOPs reduction, with an average performance drop of only 1.30\\% across multiple datasets compared to the uncompressed baseline, demonstrating strong potential for on-device deployment.

Key findings

The framework achieved a 10x parameter reduction (to 31.9M parameters) and a 6.3x FLOPs reduction, with an average performance drop of only 1.30% across multiple datasets compared to the uncompressed baseline. The compressed model even outperformed the uncompressed baseline on several out-of-domain benchmarks at 75% sparsity, suggesting that pruning acts as a structural regularization. Both cross-domain distillation and movement-guided pruning were found to be critical, especially for maintaining generalization and performance in challenging scenarios and higher compression ratios.

Approach

The proposed framework involves three steps: task-specific preparation (fine-tuning and computing movement scores), joint pruning and distillation using cross-domain knowledge distillation and movement-guided structured pruning, and backend optimization. Cross-domain knowledge distillation uses diverse unlabeled out-of-domain data to transfer generalizable forgery-discriminative representations, with representative layers selected via CKA analysis. Movement-guided structured pruning prioritizes the preservation of forgery-critical structures by leveraging fine-tuning movement scores.

Datasets

ASVspoof2019 LA (training, dev, eval), ASVspoof2021 LA, ASVspoof2021 DF, In-the-Wild, ASVspoof5, FoR

Model(s)

XLS-R (0.3B) backbone of XLSR-AASIST

Author countries

China

← Previous