FakeTracer: Catching Face-swap DeepFakes via Implanting Traces in Training

Authors: Pu Sun, Honggang Qi, Yuezun Li, Siwei Lyu

Published: 2023-07-27 02:36:13+00:00

AI Summary

FakeTracer introduces a proactive defense against face-swap DeepFakes by implanting two types of traces, sustainable (STrace) and erasable (ETrace), into training faces. During the DeepFake model's training, these manipulated faces ensure that only sustainable traces are retained in generated DeepFakes, while erasable traces are removed. This unique trace pattern allows FakeTracer to effectively expose DeepFakes by identifying their presence.

Abstract

Face-swap DeepFake is an emerging AI-based face forgery technique that can replace the original face in a video with a generated face of the target identity while retaining consistent facial attributes such as expression and orientation. Due to the high privacy of faces, the misuse of this technique can raise severe social concerns, drawing tremendous attention to defend against DeepFakes recently. In this paper, we describe a new proactive defense method called FakeTracer to expose face-swap DeepFakes via implanting traces in training. Compared to general face-synthesis DeepFake, the face-swap DeepFake is more complex as it involves identity change, is subjected to the encoding-decoding process, and is trained unsupervised, increasing the difficulty of implanting traces into the training phase. To effectively defend against face-swap DeepFake, we design two types of traces, sustainable trace (STrace) and erasable trace (ETrace), to be added to training faces. During the training, these manipulated faces affect the learning of the face-swap DeepFake model, enabling it to generate faces that only contain sustainable traces. In light of these two traces, our method can effectively expose DeepFakes by identifying them. Extensive experiments corroborate the efficacy of our method on defending against face-swap DeepFake.

Key findings

FakeTracer achieves high bit accuracy (over 90%) in recovering sustainable traces from generated DeepFake faces, indicating effective trace retention. It demonstrates competitive detection performance (0.95 D-Acc on generated faces) while maintaining high visual quality (SSIM 0.89) of the traced images. The method also shows robustness against various post-processing operations, different DeepFake model architectures, and remains effective even when only a fraction of training images are traced.

Approach

The method involves designing two imperceptible traces: a Sustainable Trace (STrace) meant to persist through the DeepFake generation process, and an Erasable Trace (ETrace) designed to be eliminated. These traces are added to the target identity's faces before they are used to train a face-swap DeepFake model. DeepFakes are then detected by verifying if a face contains STrace but lacks ETrace.

Datasets

Celeb-DF, DFDC, FaceForensics++

Model(s)

FakeTracer itself employs an STrace generator (encoder-decoder network), an STrace identifier (a two-branch network with prior and image encoders and a sub-network), and a DeepFake simulator (an auto-encoder network for face reconstruction). It targets auto-encoder-based face-swap DeepFake models, including a Celeb-DF baseline model and DeepFaceLab.

Author countries

China, USA

← Previous