ToolDF: Tool-Integrated Reasoning for Mixed-Authenticity Audio Deepfake Detection

Authors: Taewoo Kim, Young Han Lee, Nam In Park, Chanwoo Kim

Published: 2026-09-03 10:05:45+00:00

Comment: To appear in Findings of the Association for Computational Linguistics: EMNLP 2026

AI Summary

ToolDF is a tool-integrated reasoning framework for detecting mixed-authenticity audio deepfakes, where genuine and manipulated audio cues coexist. It uses an audio large language model as an orchestrator to adaptively analyze audio scenes, perform source separation when needed, route components to domain-specific experts, and aggregate evidence for an interpretable verdict. The framework introduces a new mixed-authenticity benchmark and demonstrates superior performance over monolithic baselines, along with localized, interpretable evidence.

Abstract

Audio deepfake detection is commonly formulated as clip-level binary classification of single-domain audio. However, real-world manipulated audio can exhibit mixed authenticity, where genuine and manipulated cues coexist across temporal transitions, overlapping sources, or both. This setting requires not only detecting manipulated audio but also localizing the components that provide evidence for the decision. We propose ToolDF, a tool-integrated reasoning framework for mixed-authenticity audio deepfake detection. ToolDF employs an audio large language model as an orchestrator trained with supervised tool-use trajectories. It adaptively analyzes the audio scene, selectively performs source separation, routes components to domain-specific experts, and aggregates their evidence into an interpretable verdict. We further introduce a mixed-authenticity ADD benchmark covering temporal transitions, acoustic overlaps, and hybrid mixtures. Experimental results show that ToolDF achieves the best overall performance on composite-type detection, achieving macro-F1 gains of 3.72 and 14.39 points over the strongest monolithic baseline and a fixed pipeline, respectively, while providing interpretable evidence localized to temporal regions and acoustic sources. Our source code and dataset are publicly available online.

Key findings

ToolDF achieves the best overall performance on composite-type detection, with macro-F1 gains of 3.72 points over the strongest monolithic baseline and 14.39 points over a fixed pipeline. It provides interpretable evidence localized to temporal regions and acoustic sources. Ablation studies highlight the critical role of audio understanding and planning stages for robust performance.

Approach

ToolDF employs an audio large language model (ALLM) as an orchestrator. This ALLM is trained with supervised tool-use trajectories to analyze the audio scene, selectively perform source separation, route audio components to specialized deepfake detectors (speech, singing, music, environmental sound), and aggregate their individual authenticity judgments into a final, interpretable verdict. The process involves audio understanding, tool-use planning, localized tool execution, and evidence aggregation.

Datasets

ASVspoof2019, CtrSVDD, EnvSDD, FakeMusicCaps, MusicCaps. A new mixed-authenticity ADD benchmark was constructed using these datasets, covering temporal transitions (C1), acoustic overlaps (C2), and hybrid mixtures (C3).

Model(s)

Qwen2.5-Omni-3B (as ALLM backbone), Demucs v4 (for source separation), XLSR-AASIST expert detectors (for speech, singing, music, environmental sound).

Author countries

South Korea

← Previous