Dual-Branch Gated Fusion for Open-Set Audio Deepfake Source Tracing

Authors: Awais Khan, Kutub Uddin, Khalid Malik

Published: 2026-06-08 22:22:48+00:00

AI Summary

This paper introduces a dual-branch gated fusion framework for open-set audio deepfake source tracing, aiming to attribute synthetic utterances to their originating systems even for unseen synthesizers. The framework integrates frozen XLSR-53 embeddings with a novel 66-dimensional handcrafted descriptor, CORES, using an input-conditioned gate to adaptively weight their contributions. Jointly trained with multiple loss functions, the system achieves 97.6% in-domain accuracy and significantly reduces false positive rates for out-of-distribution detection on the MLAAD benchmark.

Abstract

Attributing a synthetic utterance to its originating system remains an open challenge: closed-set models fail to reject unseen synthesizers and produce overconfident predictions. To address this, we propose a dual-branch gated fusion framework that pairs XLSR-53 with CORES, a 66-dimensional descriptor that, unlike prior Linear Filter Bank (LFB)-only work, spans cepstral, oscillatory, rhythmic, energy, and spectral dimensions to capture complementary synthesis artifacts. Our analysis shows XLSR-53 remains discriminative in-domain (ID) while CORES generalizes stably under distribution shift (OOD), yet their naive concatenation fails due to SSL representational imbalance. To resolve this, an input-conditioned gate adaptively weights each branch under joint training with cross-entropy, an energy margin loss for ID/OOD separation, and a gate diversity term. On the MLAAD benchmark, our system achieves 97.6\\% ID accuracy, 4.9\\% EERc, and an 83.5\\% relative FPR95 reduction over the Interspeech 2025 baseline.

Key findings

The system achieved 97.6% in-domain accuracy, 4.9% EERc, and a 10.4% FPR95 on the MLAAD benchmark, representing an 83.5% relative FPR95 reduction over the Interspeech 2025 baseline. It significantly outperforms previous state-of-the-art models in open-set detection (OOD accuracy and EER) with substantially fewer parameters (0.9M). Ablation studies confirmed that the adaptive gating mechanism is crucial for resolving the trade-off between in-domain classification and out-of-distribution rejection.

Approach

The proposed method employs a dual-branch gated fusion framework where one branch processes frozen XLSR-53 embeddings and the other utilizes CORES, a 66-dimensional handcrafted descriptor capturing diverse synthesis artifacts. An input-conditioned lightweight gating network adaptively weights the contributions of these two branches. The system is trained using a combination of cross-entropy loss for in-domain classification, an energy margin loss for ID/OOD separation, and a gate diversity term to prevent branch collapse.

Datasets

MLAAD source tracing protocol, MUSAN (for additive noise augmentation), RIRs (for reverberation augmentation)

Model(s)

XLSR-53 (frozen encoder), custom handcrafted descriptor (CORES), and a dual-branch neural network with an input-conditioned gating mechanism

Author countries

USA

← Previous