Diff-ID: An Explainable Identity Difference Quantification Framework for DeepFake Detection

View on arXiv ← Back to list

Authors: Chuer Yu, Xuhong Zhang, Yuxuan Duan, Senbo Yan, Zonghui Wang, Yang Xiang, Shouling Ji, Wenzhi Chen

Published: 2023-03-30 10:10:20+00:00

AI Summary

Diff-ID is a novel deepfake detection approach that quantifies identity loss caused by facial manipulations. It uses a face-swapping generator to align authentic and test images, visualizing and measuring identity differences to distinguish real from fake images.

Abstract

Despite the fact that DeepFake forgery detection algorithms have achieved impressive performance on known manipulations, they often face disastrous performance degradation when generalized to an unseen manipulation. Some recent works show improvement in generalization but rely on features fragile to image distortions such as compression. To this end, we propose Diff-ID, a concise and effective approach that explains and measures the identity loss induced by facial manipulations. When testing on an image of a specific person, Diff-ID utilizes an authentic image of that person as a reference and aligns them to the same identity-insensitive attribute feature space by applying a face-swapping generator. We then visualize the identity loss between the test and the reference image from the image differences of the aligned pairs, and design a custom metric to quantify the identity loss. The metric is then proved to be effective in distinguishing the forgery images from the real ones. Extensive experiments show that our approach achieves high detection performance on DeepFake images and state-of-the-art generalization ability to unknown forgery methods, while also being robust to image distortions.

Key findings

Diff-ID achieves state-of-the-art generalization to unseen forgery methods and robustness to image distortions. It outperforms other methods on multiple datasets, particularly CelebDF and DFo. The approach's effectiveness is demonstrated across various face-swapping generators.

Approach

Diff-ID aligns authentic and test images using a face-swapping generator, creating attribute-aligned versions. It then calculates a custom metric quantifying pixel-level identity differences between these aligned images to classify images as real or fake.

Datasets

FaceForensics++, Google DeepFake Detection (DFD), CelebDF, DeeperForensics-1.0 (DFo), CelebA

Model(s)

SimSwap (face-swapping generator), ArcFace (face recognition model), optionally FaceShifter and HiFiFace

Author countries

China, USA, Australia

← Previous