LatentForensics: Towards frugal deepfake detection in the StyleGAN latent space

View on arXiv ← Back to list

Authors: Matthieu Delmas, Renaud Seguier

Published: 2023-03-30 08:36:48+00:00

AI Summary

This paper proposes a deepfake detection method operating in the latent space of StyleGAN, a generative adversarial network. This approach leverages StyleGAN's latent space structure to create a lightweight binary classification model that outperforms state-of-the-art methods, particularly when training data is scarce.

Abstract

The classification of forged videos has been a challenge for the past few years. Deepfake classifiers can now reliably predict whether or not video frames have been tampered with. However, their performance is tied to both the dataset used for training and the analyst's computational power. We propose a deepfake detection method that operates in the latent space of a state-of-the-art generative adversarial network (GAN) trained on high-quality face images. The proposed method leverages the structure of the latent space of StyleGAN to learn a lightweight binary classification model. Experimental results on standard datasets reveal that the proposed approach outperforms other state-of-the-art deepfake classification methods, especially in contexts where the data available to train the models is rare, such as when a new manipulation method is introduced. To the best of our knowledge, this is the first study showing the interest of the latent space of StyleGAN for deepfake classification. Combined with other recent studies on the interpretation and manipulation of this latent space, we believe that the proposed approach can further help in developing frugal deepfake classification methods based on interpretable high-level properties of face images.

Key findings

The StyleGAN latent space proved superior to PCA for dimensionality reduction in deepfake detection. The proposed method achieved competitive accuracy with significantly lower computational cost compared to state-of-the-art methods, especially when training data is limited. The method outperforms other models on smaller datasets, suggesting robustness to novel deepfake techniques.

Approach

The method projects face images into the latent space of a pre-trained StyleGAN using an encoder-based inversion method (like E2Style). A lightweight multi-layer perceptron is then trained on these lower-dimensional latent vectors to classify images as real or fake.

Datasets

Deepfake Detection Challenge (DFDC) preview dataset, CelebDF v1 and v2

Model(s)

Multilayer Perceptron (MLP) with 5 fully-connected layers; StyleGAN and E2Style are used for latent space projection, but are not trained in this work. Comparisons are made against MesoNet, XceptionNet, and EfficientNet.

Author countries

France

← Previous