OGAN: Disrupting Deepfakes with an Adversarial Attack that Survives Training

Authors: Eran Segalis, Eran Galili

Published: 2020-06-17 17:18:29+00:00

Comment: 10 pages

AI Summary

This paper introduces OGAN (Oscillating GAN), a novel training-resistant adversarial attack designed to disrupt face-swapping deepfakes. OGAN generates spatial-temporal distortions in deepfake outputs, even if the adversarial images are included in the deepfake model's training set, thus complementing existing deepfake detection efforts with a prevention mechanism. It frames the problem as a bilevel optimization to train a generator against a face-swapping model.

Abstract

Recent advances in autoencoders and generative models have given rise to effective video forgery methods, used for generating so-called deepfakes. Mitigation research is mostly focused on post-factum deepfake detection and not on prevention. We complement these efforts by introducing a novel class of adversarial attacks---training-resistant attacks---which can disrupt face-swapping autoencoders whether or not its adversarial images have been included in the training set of said autoencoders. We propose the Oscillating GAN (OGAN) attack, a novel attack optimized to be training-resistant, which introduces spatial-temporal distortions to the output of face-swapping autoencoders. To implement OGAN, we construct a bilevel optimization problem, where we train a generator and a face-swapping model instance against each other. Specifically, we pair each input image with a target distortion, and feed them into a generator that produces an adversarial image. This image will exhibit the distortion when a face-swapping autoencoder is applied to it. We solve the optimization problem by training the generator and the face-swapping model simultaneously using an iterative process of alternating optimization. Next, we analyze the previously published Distorting Attack and show it is training-resistant, though it is outperformed by our suggested OGAN. Finally, we validate both attacks using a popular implementation of FaceSwap, and show that they transfer across different target models and target faces, including faces the adversarial attacks were not trained on. More broadly, these results demonstrate the existence of training-resistant adversarial attacks, potentially applicable to a wide range of domains.

Key findings

OGAN successfully creates significant artifacts in deepfaked videos, demonstrating strong training resistance even when adversarial samples are included in the deepfake model's training set, outperforming the previously published Distorting Attack in this scenario. Both OGAN and the Distorting Attack show transferability across different face-swapping architectures (dfl-h128, dfl-sae, realface) and target faces. The attacks remain effective even after target deepfake models are trained for extended iterations.

Approach

OGAN solves a bilevel optimization problem by training an adversarial generator and a face-swapping model instance against each other. The generator takes an input image and a target distortion to produce an adversarial image, which will exhibit the target distortion when a face-swapping autoencoder is applied. This is achieved through an iterative process of alternating optimization.

Datasets

Videos collected from YouTube for 3 people (A, B, C) were used to create datasets DA, DB, and DC, each containing approximately 5000 face images. A subset PA of 444 images from one video of person A was used as controlled data.

Model(s)

UNKNOWN

Author countries

UNKNOWN

← Previous