Mitigating Proxy-to-Wild Domain Gap in Deepfake Speech

Authors: Xuanjun Chen, Yun-Shing Wu, Wei-Chung Lu, Claire Lin, Haibin Wu, Hung-yi Lee, Jyh-Shing Roger Jang

Published: 2026-06-05 17:48:46+00:00

Comment: Work in progress

AI Summary

This paper addresses the proxy-to-wild domain gap in deepfake speech detection, particularly for neural audio codec-based generation (CodecFake), where models trained on proxy data struggle with generalization. The authors propose Domain-Shift Feature Augmentation (DSFA) to simulate real-world variations by perturbing feature statistics during fine-tuning. They also introduce the CoSG ExtEval dataset to provide a more challenging benchmark for evaluating generalization across diverse unseen generative models.

Abstract

Recent neural audio codec-based speech generation (CodecFake) produces highly realistic audio, posing a challenge to existing deepfake countermeasure models. While using codec resynthesized speech (CoRS) as proxy data improves performance, it often suffers from limited generalization. We propose Domain-Shift Feature Augmentation (DSFA), which simulates in-the-wild variations by transforming deterministic feature statistics into stochastic distributions during fine-tuning. To evaluate generalization, we further introduce Codec-based Speech Generation Extension Evaluation (CoSG ExtEval) dataset, a more challenging extension of the CoSG Eval (from CodecFake+) dataset, featuring 40 unseen generative models and long-form audio. Experimental results demonstrate that combining a post-trained SSL backbone with DSFA effectively narrows the proxy-to-wild domain gap. This approach achieves state-of-the-art performance across diverse CodecFake attacks in both CoSG Eval and CoSG ExtEval.

Key findings

The proposed DSFA method, combined with a post-trained SSL backbone, effectively narrows the proxy-to-wild domain gap. This approach achieves state-of-the-art performance, demonstrating improved robustness and generalization across diverse CodecFake attacks in both CoSG Eval and the more challenging CoSG ExtEval datasets. DSFA promotes domain-invariant features by improving the overlap of feature statistics distributions in the latent space.

Approach

The approach leverages a post-trained Self-Supervised Learning (SSL) backbone for robust feature extraction and introduces Domain-Shift Feature Augmentation (DSFA). DSFA simulates in-the-wild variations by transforming deterministic feature statistics (mean and standard deviation) into stochastic distributions through batch-wise statistical perturbations and Adaptive Instance Normalization (AdaIN) during fine-tuning. A joint supervised contrastive and cross-entropy loss is used for training.

Datasets

CodecFake+ (including CoRS for training), CoSG Eval, CoSG ExtEval. VCTK corpus (for CoRS generation), ASVspoof19 LA (for baseline comparisons).

Model(s)

Wav2Vec2-Large-AntiDeepfake (post-trained SSL backbone).

Author countries

Taiwan

← Previous