HQ-MPSD: A Multilingual Artifact-Controlled Benchmark for Partial Deepfake Speech Detection

View on arXiv ← Back to list

Authors: Menglu Li, Majd Alber, Ramtin Asgarianamiri, Lian Zhao, Xiao-Ping Zhang

Published: 2025-12-15 06:18:43+00:00

AI Summary

This paper introduces HQ-MPSD, a high-quality, multilingual partial deepfake speech dataset spanning 350.8 hours across eight languages, designed to serve as a challenging benchmark. The dataset minimizes superficial splicing artifacts using linguistically coherent splice points derived from forced alignment and incorporates real-world background effects. Benchmarking results show that state-of-the-art detection models suffer severe performance degradation (over 80% drops), confirming the difficulty and realism introduced by HQ-MPSD.

Abstract

Detecting partial deepfake speech is challenging because manipulations occur only in short regions while the surrounding audio remains authentic. However, existing detection methods are fundamentally limited by the quality of available datasets, many of which rely on outdated synthesis systems and generation procedures that introduce dataset-specific artifacts rather than realistic manipulation cues. To address this gap, we introduce HQ-MPSD, a high-quality multilingual partial deepfake speech dataset. HQ-MPSD is constructed using linguistically coherent splice points derived from fine-grained forced alignment, preserving prosodic and semantic continuity and minimizing audible and visual boundary artifacts. The dataset contains 350.8 hours of speech across eight languages and 550 speakers, with background effects added to better reflect real-world acoustic conditions. MOS evaluations and spectrogram analysis confirm the high perceptual naturalness of the samples. We benchmark state-of-the-art detection models through cross-language and cross-dataset evaluations, and all models experience performance drops exceeding 80% on HQ-MPSD. These results demonstrate that HQ-MPSD exposes significant generalization challenges once low-level artifacts are removed and multilingual and acoustic diversity are introduced, providing a more realistic and demanding benchmark for partial deepfake detection. The dataset can be found at: https://zenodo.org/records/17929533.

Key findings

HQ-MPSD achieves the highest perceptual naturalness (MOS 3.68) among compared partial deepfake datasets by successfully reducing audible and visual boundary artifacts. Cross-language evaluation showed drastic performance drops across all SOTA models when trained on English and tested on seven unseen languages. Cross-dataset evaluation confirmed that models trained on legacy data like PartialSpoof suffered performance collapse (EER increase up to 90%) when tested on the artifact-controlled HQ-MPSD, proving their reliance on dataset-specific cues.

Approach

The authors developed a high-quality dataset generation pipeline focused on eliminating acoustic artifacts. This includes pre-normalizing bonafide and deepfake segments, using word-level forced alignment to ensure linguistically coherent replacement boundaries, and applying acoustic augmentation (noise and RIR) to simulate diverse real-world conditions. This artifact-controlled approach aims to force detection models to learn genuine synthesis cues rather than superficial splicing flaws.

Datasets

HQ-MPSD (Proposed), Multilingual LibriSpeech (MLS), PartialSpoof, Half-Truth. Augmentation sources: OpenSLR 26 (RIR), MUSAN (Noise).

Model(s)

GAT-ST, TDAM, Nes2Net. Feature extractors include SincNet, MFCC, Spectrogram, and W2v2-XLSR.

Author countries

China, Canada

← Previous