On the Robustness of Audio Deepfake Detection under Audio Watermarking

Authors: Zi Qian Yong, Ajinkya Kulkarni, Julia Lau, Hwa Hui Tew, Shu Min Leong, Raphael Phan, Sébastien Marcel

Published: 2026-08-25 07:18:16+00:00

AI Summary

This research investigates the robustness of audio deepfake detection (ADD) systems under audio watermarking, treating watermarking as a structured, non-adversarial perturbation. The study evaluates multiple ADD models across various datasets, analyzing performance degradation and embedding-space shifts caused by watermarks. Key findings indicate a strong dataset-dependent impact, with significant performance drops on some datasets and large embedding shifts correlating with severe detection degradation.

Abstract

Recent advances in generative audio models have enabled highly realistic synthetic speech, increasing the importance of reliable audio deepfake detection (ADD) systems. While prior studies have primarily focused on adversarially optimized perturbations, the robustness of ADD systems under realistic signal transformations remains insufficiently understood. In this work, we investigate the impact of audio watermarking on ADD systems by treating watermarking as a structured, non-adversarial perturbation rather than a conventional attack mechanism. Using a watermark-based evaluation framework built upon WavMark, we evaluate multiple self-supervised learning (SSL), Convolutional Neural Network (CNN) and Graph Neural Netrowk (GNN)-based ADD models across several benchmark datasets. Beyond conventional detection metrics, we further analyze watermark-induced representation shifts using Fréchet Distance, cosine similarity, and L2 distance in the embedding space. Experimental results reveal a strong dataset-dependent behavior: watermarking causes substantial performance degradation on ASVspoof 2021 LA and DF, while exhibiting limited impact on ASVspoof 2024, FoR, and ITW. Moreover, large embedding-space shifts are strongly associated with severe detection degradation, suggesting that watermark-induced perturbations can substantially alter the feature representations relied upon by current ADD systems. These findings demonstrate that benign signal transformations designed for content protection can expose previously overlooked robustness vulnerabilities in audio deepfake detection systems. Our code is available at https://github.com/ziqian0925/wm-ADD-robustness.git

Key findings

Watermarking causes substantial performance degradation on ASVspoof 2021 LA and DF, while having limited impact on ASVspoof 2024, FoR, and ITW, indicating a strong dataset-dependent behavior. Large embedding-space shifts, particularly for spoof samples in SSL-based models, are strongly associated with severe detection degradation, suggesting that watermark-induced perturbations can significantly alter the feature representations relied upon by ADD systems.

Approach

The authors propose a watermark-based evaluation framework built on WavMark to assess ADD systems. They embed watermarks into audio samples and then feed these watermarked audios to various pre-trained ADD models to measure detection performance (EER) and analyze representation-level shifts using Fréchet Distance, cosine similarity, and L2 distance in the embedding space.

Datasets

ASVspoof 2021 LA, ASVspoof 2021 DF, ASVspoof 2024, In-the-Wild (ITW), Fake-or-Real (FoR)

Model(s)

XLSR+SLS, TCM, Nes2Net, AASIST, RawGAT-ST, RawNet2, RawTFNet (and WavMark for watermarking)

Author countries

Malaysia, Switzerland

← Previous