EEG-Features for Generalized Deepfake Detection

View on arXiv ← Back to list

Authors: Arian Beckmann, Tilman Stephani, Felix Klotzsche, Yonghao Chen, Simon M. Hofmann, Arno Villringer, Michael Gaebler, Vadim Nikulin, Sebastian Bosse, Peter Eisert, Anna Hilsmann

Published: 2024-05-14 12:06:44+00:00

AI Summary

This study introduces a novel Deepfake detection approach utilizing electroencephalography (EEG) data recorded from a human participant viewing real and manipulated facial images from the FaceForensics++ dataset. EEG measurements serve as input features to a binary Support Vector Classifier tasked with discriminating between real and Deepfake stimuli. Preliminary results demonstrate that human neural processing signals can be effectively integrated into Deepfake detection frameworks, suggesting the potential for a generalized neural representation of artifacts in computer-generated faces.

Abstract

Since the advent of Deepfakes in digital media, the development of robust and reliable detection mechanism is urgently called for. In this study, we explore a novel approach to Deepfake detection by utilizing electroencephalography (EEG) measured from the neural processing of a human participant who viewed and categorized Deepfake stimuli from the FaceForensics++ datset. These measurements serve as input features to a binary support vector classifier, trained to discriminate between real and manipulated facial images. We examine whether EEG data can inform Deepfake detection and also if it can provide a generalized representation capable of identifying Deepfakes beyond the training domain. Our preliminary results indicate that human neural processing signals can be successfully integrated into Deepfake detection frameworks and hint at the potential for a generalized neural representation of artifacts in computer generated faces. Moreover, our study provides next steps towards the understanding of how digital realism is embedded in the human cognitive system, possibly enabling the development of more realistic digital avatars in the future.

Key findings

The study successfully demonstrated that EEG features can be used for Deepfake detection, even for out-of-domain fakes not seen during training, with classification performances above chance level. This suggests that human neural processing may contain a generalized representation of artificiality in computer-generated faces. The findings highlight the potential for integrating human cognitive responses into robust Deepfake detection frameworks.

Approach

The authors collected electroencephalography (EEG) data from a human observer while they viewed real and Deepfake facial images from the FaceForensics++ dataset. Features were extracted from the pre-processed EEG signals, undergoing dimensionality reduction via PCA or ICA. A binary Support Vector Classifier (SVC) was then trained on these EEG features to distinguish between real and manipulated faces, also evaluating its generalization capabilities to unseen Deepfake generation methods.

Datasets

FaceForensics++ (specifically Deepfakes (DF) and FaceSwap (FS) methods)

Model(s)

Support Vector Classifier (SVC) with Principal Component Analysis (PCA) and Independent Component Analysis (ICA) for dimensionality reduction.

Author countries

Germany

← Previous