ReplayFactor: Replay-Aware Dual-Attribute Learning for Practical Speech Deepfake Detection

Authors: Miao He, Jiayi Zhou, Yu Zheng, Zhongjie Ba, Peng Cheng, Yuxi Zhou, Junjie Zhu, Changhao Zhang, Jian Liu

Published: 2026-11-02 06:53:41+00:00

Comment: 5 pages, 3 figures. Submitted to ICASSP 2027

AI Summary

This paper introduces ReplayFactor, a dual-attribute learning framework for robust speech deepfake detection, particularly addressing performance degradation on physically replayed deepfakes. It enhances replay evidence by fusing self-supervised learning representations with an acoustic prior and employs multi-task and supervised contrastive learning for generalizable representations. The authors also contribute ReplayFake-CN, a large-scale Chinese dataset of replayed synthetic utterances.

Abstract

Advances in synthesis techniques have made deepfake speech increasingly realistic, amplifying voice security threats, including unauthorized voice cloning and impersonation. Although existing deepfake detectors perform well on digitally generated speech, their performance often degrades on physically replayed deepfakes, which constitute a low-cost and practical threat to real-world systems. Existing research focuses primarily on dataset construction and evaluation, while available dataset diversity and replay-aware detector design remain limited. We propose ReplayFactor, a dual-attribute framework that enhances replay evidence by fusing self-supervised learning (SSL) representations with an acoustic prior, while leveraging multi-task learning and supervised contrastive learning to promote robust, generalizable representations. We also construct ReplayFake-CN, a large-scale, multi-device Chinese dataset comprising approximately 185k replayed synthetic utterances to complement public benchmarks. Our framework improves macro-F1 from 60.99% to 76.44% and reduces EER from 11.86% to 3.05% on the EchoFake open set over the baseline, demonstrating strong potential for practical replayed deepfake detection.


Key findings
ReplayFactor significantly improves performance on open-set scenarios, reducing EER from 11.86% to 3.05% and improving macro-F1 from 60.99% to 76.44% on the EchoFake open set compared to the baseline. It consistently outperforms other methods on the ReplayFake-CN dataset, especially under unseen synthesis models, demonstrating improved generalization to diverse replay conditions and synthesis methods. The acoustic prior and supervised contrastive learning components were found to be crucial for enhancing replay-sensitive evidence and maintaining attribute consistency.
Approach
ReplayFactor utilizes a dual-attribute framework. It fuses self-supervised learning (SSL) representations, processed by an AASIST graph backend, with an acoustic prior (CQCCs encoded by an LCNN) through a gated mechanism to enhance replay-sensitive cues. The framework then employs multi-task learning for four-class classification (bona fide, fake, replayed bona fide, replayed fake) and two auxiliary attribute tasks (source and replay), combined with dual-space supervised contrastive learning to promote robust, generalizable representations.
Datasets
ReplayFake-CN (newly constructed), EchoFake, ReplayDF
Model(s)
MMS-300M (as SSL encoder), AASIST (graph backend), LCNN (for acoustic prior encoding)
Author countries
China