RealComm: Benchmarking and Adapting Audio Deepfake Detection over Real Communication Channels

View on arXiv ← Back to list

Authors: Li Wang, Jindong Wang, Wan Lin, Kunyu Feng, Yufan Lin, Lei Wang, Rizhao Cai, Ce Fang, Jinzhe Xue, Jie Shi, Haizhou Li, Zhizheng Wu

Published: 2026-10-20 01:19:17+00:00

Comment: 12 pages, 9 figures, and 3 tables. Submitted to IEEE Transactions on Audio, Speech and Language Processing

AI Summary

This paper introduces RealComm, a dataset for benchmarking audio deepfake detection over real mobile communication channels, and RealComm-Aug, a waveform augmentation pipeline. It demonstrates significant degradation of existing deepfake detectors on call-transmitted speech, especially with acoustic injection. The study shows that combining RealComm-Aug with real-call training substantially improves detection performance in the call domain.

Abstract

Acoustic capture, device processing, and transmission coding in real calls jointly alter speech signals and may affect the reliability of audio deepfake detection. To systematically evaluate these effects, we introduce RealComm, a dataset pairing digital speech with recordings of the same utterances transmitted through real mobile calls. The acquisition design covers acoustic and wired injection, handset and speakerphone operation, and caller-side denoising, with bona fide and synthetic speech recorded under matched conditions. The dataset includes RealCommBench for evaluation and RealCommTrain for adaptation. We further develop RealComm-Aug, a staged waveform augmentation pipeline for simulating acquisition and transmission effects, and investigate its combination with real-call training. Experiments with existing detectors reveal substantial degradation after call transmission, even for models with strong digital-domain performance; acoustic injection is particularly challenging. Paired quality and spectral analyses show that quality degradation is associated with poorer detection across recording conditions for the evaluated detectors, while high-frequency attenuation alone does not explain detection difficulty. Simulated augmentation and real-call training provide complementary benefits, with their combination achieving the lowest call-domain error for each of three evaluated detectors. For Teffic-Audio, the equal error rate decreases from 37.94% to 22.37%. The demo and benchmark access information are provided through the project repository.

Key findings

Existing audio deepfake detectors suffer substantial performance degradation (over 30 EER points) when speech is transmitted through real communication channels, with acoustic injection being particularly challenging. Quality deterioration of speech, as measured by WER, CER, SIM, and DNSMOS, consistently correlates with detection degradation. Combining simulated augmentation (RealComm-Aug) with real-call training achieves the lowest call-domain error for all evaluated detectors, significantly improving performance (e.g., Teffic-Audio EER from 37.94% to 22.37%).

Approach

The authors introduce a dataset, RealComm, which pairs digital speech with recordings transmitted through real mobile calls, covering various injection methods, device operations, and denoising settings. They evaluate existing deepfake detectors on this dataset and develop RealComm-Aug, a staged waveform augmentation pipeline to simulate channel effects. They then investigate combining RealComm-Aug with real-call training for detector adaptation.

Datasets

RealComm (RealCommBench for evaluation, RealCommTrain for adaptation), ASVspoof 2021 LA, ASVspoof 5, In-the-Wild, Seed-TTS Eval, RIRS_NOISES, Graphi07, MUSAN, micirp, MADIR, POLIPHONE

Model(s)

WavLM-L (Teffic), Whisper (Teffic), AASIST (Teffic), Teffic-Audio, DF Arena 500M V1, XLSR-SLS

Author countries

China

← Previous