XAI-Grounded Explanation Generation for Speech Deepfake Detection with Training-Free Multimodal Large Language Models

Authors: Yupei Li, Qiyang Sun, Xiaoliang Wu, Chenxi Wang, Berrak Sisman, Björn W. Schuller

Published: 2026-06-15 02:55:21+00:00

Comment: Accepted at Interspeech 2026

AI Summary

This paper introduces a training-free framework that integrates traditional explainable AI (XAI) evidence with multimodal Large Language Models (LLMs) to generate grounded and specific natural language explanations for Speech Deepfake Detection (SDD). The approach aims to address the limitations of generic LLM explanations and low-level XAI outputs, enhancing the trustworthiness and interpretability of SDD systems. The authors also construct and publicly release a large-scale explainable SDD dataset based on the PartialSpoof dataset.

Abstract

Speech deepfake detection (SDD) systems require trustworthy explanations for reliable decision-making. Existing explanation ways mainly fall into two categories. Traditional explainable AI (XAI), such as gradient-based attribution, produces low-level attribution signals tightly coupled with model decisions, and harder to be understood by human than natural language explanations. Meanwhile, large language model (LLM)-based explanation generation often produces generic and ungrounded descriptions due to the lack of heuristic evidence and task-specific supervision, stemming from limited grounded explanation datasets for SDD. We therefore propose a training-free explanation framework that integrates XAI evidence with multimodal LLMs to generate grounded and specific explanations. Using the PartialSpoof dataset, we construct a grounded explanation dataset and show that methods with XAI increase inside accuracy by over 45\\%, verified through human evaluation and faithfulness checks.

Key findings

XAI-guided explanation generation significantly improved alignment with human subjective assessments in correctness, evidence support, and specificity, notably reducing hallucinations. Quantitative analysis showed that XAI-guided methods achieved higher Inside Accuracy (IA) for abnormal time period detection and identified more informative regions with substantially higher Area-Normalised Local Logit Sensitivity. The study also created and released a large-scale explainable SDD dataset with approximately 65,000 explanation instances.

Approach

The proposed XAI-Grounded Explanation Generation via LLMs (XGEG) framework uses pre-trained SDD models (HuBERT, Wav2Vec 2.0, WavLM) and acoustic features (openSMILE processed by an MLP with SHAP) to generate XAI evidence. Spectrogram-based XAI (IG, LIME, Saliency) are summarized by a vision-LLM (Qwen2.5-VL-7B), and these summaries, along with SHAP-derived feature importances, are then fed into a multimodal LLM (Qwen3-Omni-30B) to produce structured, human-understandable textual explanations.

Datasets

PartialSpoof dataset

Model(s)

HuBERT, Wav2Vec 2.0, WavLM (for SDD foundation models); four-layer Multilayer Perceptron (MLP) (for SHAP analysis); Qwen2.5-VL-7B (vision-LLM); Qwen3-Omni-30B (multimodal LLM)

Author countries

United Kingdom, Germany, United Arab Emirates, United States of America

← Previous