Component-Level Ensemble Fusion for Speech and Environmental Sound Deepfake Detection

Authors: André Runewicz, Karla Schäfer, Martin Steinebach

Published: 2026-07-17 14:00:32+00:00

Comment: Accepted to 2026 ICME workshop

AI Summary

This paper presents a component-level ensemble system for the ESDD2 challenge on environment-aware speech and sound deepfake detection. The approach fine-tunes four pre-trained anti-spoofing models with three binary heads (original, speech, environmental sound detection) and combines them using margin-space score fusion and bias calibration. This system achieved a macro-F1 of 0.7828 on the test set, ranking 5th in the challenge and significantly outperforming the baseline.

Abstract

This paper describes our submission to the ICME 2026 ESDD2 challenge on environment-aware speech and sound deepfake detection. The task requires five-class classification of audio clips in which speech, environmental sound, both components, or neither component may be spoofed. We propose a component-level ensemble system based on four publicly available pre-trained anti-spoofing models: XLSR-Mamba, DF-Arena, SLS, and TCM-ADD. Each model is fine-tuned on the official CompSpoofV2 development data using three binary heads for original, speech, and environmental sound detection. We further train RawBoost-augmented variants and combine selected checkpoints using margin-space score fusion. A component-wise fusion strategy with lightweight head- and class-bias calibration yields our best configuration, reaching 0.7715 macro-F1 on the evaluation set and 0.7828 macro-F1 on the test set, ranking 5th out of 31 teams in the final ranking phase and substantially outperforming the official baseline.

Key findings

The component-wise fusion strategy with head- and class-bias calibration significantly improved macro-F1 scores, outperforming individual models and conservative ensemble approaches. RawBoost augmentation showed model-dependent benefits, indicating the importance of including both augmented and non-augmented variants. The system achieved strong performance in the ESDD2 challenge, especially for environmental sound detection, despite a slightly worse speech EER compared to the baseline.

Approach

The system employs a component-level ensemble strategy, fine-tuning four pre-trained anti-spoofing models (XLSR-Mamba, DF-Arena, SLS, TCM-ADD) on the CompSpoofV2 dataset. Each model is adapted with three binary classification heads to detect original, spoofed speech, and spoofed environmental sound. RawBoost augmentation variants are also trained, and selected checkpoints are combined using margin-space score fusion, followed by head- and class-bias calibration.

Datasets

CompSpoofV2

Model(s)

XLSR-Mamba, DF-Arena, SLS, TCM-ADD

Author countries

Germany

← Previous