FaceSigns: Semi-Fragile Neural Watermarks for Media Authentication and Countering Deepfakes

Authors: Paarth Neekhara, Shehzeen Hussain, Xinqiao Zhang, Ke Huang, Julian McAuley, Farinaz Koushanfar

Published: 2022-04-05 03:29:30+00:00

Comment: 13 pages, 8 figures

AI Summary

This paper introduces FaceSigns, a deep learning-based semi-fragile watermarking technique for media authentication and countering Deepfakes. It proactively embeds an invisible secret message into real images, designed to be robust to benign image processing operations but fragile to malicious facial manipulations. FaceSigns reliably detects manipulated content with a high AUC score of 0.996, significantly outperforming prior watermarking and steganography techniques for deepfake detection.

Abstract

Deepfakes and manipulated media are becoming a prominent threat due to the recent advances in realistic image and video synthesis techniques. There have been several attempts at combating Deepfakes using machine learning classifiers. However, such classifiers do not generalize well to black-box image synthesis techniques and have been shown to be vulnerable to adversarial examples. To address these challenges, we introduce a deep learning based semi-fragile watermarking technique that allows media authentication by verifying an invisible secret message embedded in the image pixels. Instead of identifying and detecting fake media using visual artifacts, we propose to proactively embed a semi-fragile watermark into a real image so that we can prove its authenticity when needed. Our watermarking framework is designed to be fragile to facial manipulations or tampering while being robust to benign image-processing operations such as image compression, scaling, saturation, contrast adjustments etc. This allows images shared over the internet to retain the verifiable watermark as long as face-swapping or any other Deepfake modification technique is not applied. We demonstrate that FaceSigns can embed a 128 bit secret as an imperceptible image watermark that can be recovered with a high bit recovery accuracy at several compression levels, while being non-recoverable when unseen Deepfake manipulations are applied. For a set of unseen benign and Deepfake manipulations studied in our work, FaceSigns can reliably detect manipulated content with an AUC score of 0.996 which is significantly higher than prior image watermarking and steganography techniques.

Key findings

FaceSigns successfully embeds a 128-bit secret as an imperceptible watermark, achieving high bit recovery accuracy under benign transformations while being non-recoverable under unseen Deepfake manipulations. It reliably detects manipulated content with an AUC score of 0.996, demonstrating superior performance over previous image watermarking and steganography methods. The system also generalizes effectively to images containing multiple faces.

Approach

FaceSigns employs an end-to-end deep learning framework comprising an encoder, decoder, and adversarial discriminator. The encoder embeds a secret message into an image, making it imperceptible. The framework is trained to ensure the message is recoverable after benign image transformations but unrecoverable after malicious facial manipulations like Deepfakes, thereby enabling authentication and detection of tampering.

Datasets

CelebA, Celebrity Together dataset (for multiple faces testing)

Model(s)

UNKNOWN

Author countries

USA

← Previous