The Watermark Shortcut: How Provenance Marking Sabotages Audio Deepfake Detection

Authors: Nicolas M. Müller, Pascal Debus

Published: 2026-06-22 13:42:31+00:00

AI Summary

This research reveals a critical vulnerability in audio deepfake detection: when synthetic speech is prominently watermarked and human speech is not, detectors learn a 'watermark => fake' shortcut, leading to generalization degradation, strip-to-evade attacks, and mark-to-frame false positives. The authors demonstrate these failures in white-box and black-box experiments and propose a mitigation involving retraining with watermarks on both genuine and synthetic speech.

Abstract

Provenance watermarking is increasingly treated as a safeguard for synthetic speech, whether built directly into speech-generation models such as Chatterbox, provided through dedicated techniques such as AudioSeal, or deployed by commercial platforms such as ElevenLabs. We identify a previously uncharacterized liability: when synthetic speech is watermarked and human speech is not, detectors trained alongside latch onto the watermark as a spurious watermark => fake shortcut. This single feature yields three coupled failures: generalization degradation (model performance deteriorates on unseen data), strip-to-evade (a watermarked fake escapes once unwatermarked), and mark-to-frame (watermarking a real voice flags it as fake). In a controlled white-box experiment, a watermark-trained detector shows all three (for example, mark-to-frame lifts Equal Error Rate from 16% to 75%). In a black-box test of a commercial API, we show that adding a watermark to real speech disguises it as fake. However, this shortcut is fixable: retraining with the watermark on both classes decorrelates it and restores clean behavior. We release experiment data as a paired clean-versus-watermarked corpus (WASP).

Key findings

Detectors trained with watermarked synthetic speech and unwatermarked human speech learn a 'watermark => fake' shortcut, resulting in significantly degraded generalization, successful evasion by removing watermarks, and misclassification of real, watermarked speech as fake. This shortcut is fixable by retraining the detector with watermarks applied to both genuine and synthetic audio, effectively decorrelating the watermark from the authenticity label and restoring robust detection performance.

Approach

The study investigates shortcut learning in audio deepfake detectors when provenance watermarking is applied selectively. They perform a controlled white-box experiment by training two AASIST detectors on ASVspoof19, one with watermarked synthetic speech and one clean. A black-box experiment queries a commercial API using their newly created WASP corpus to test for similar vulnerabilities. The solution involves retraining with watermarks applied to both synthetic and real audio.

Datasets

ASVspoof19 LA, ASVspoof2021-LA, In-the-Wild, M-AILABS, AISHELL-3, WASP corpus (newly created).

Model(s)

AASIST

Author countries

Germany

← Previous