Adversarially robust deepfake media detection using fused convolutional neural network predictions

Authors: Sohail Ahmed Khan, Alessandro Artusi, Hang Dai

Published: 2021-02-11 11:28:00+00:00

AI Summary

This paper proposes an adversarially robust deepfake media detection system using a fusion of predictions from three deep Convolutional Neural Networks (VGG16, InceptionV3, XceptionNet). The approach classifies fake and real images extracted from videos, aiming to improve generalization and robustness against unseen data and adversarial attacks. The fused model outperforms state-of-the-art methods and demonstrates resilience when individual models are compromised by adversarial attacks.

Abstract

Deepfakes are synthetically generated images, videos or audios, which fraudsters use to manipulate legitimate information. Current deepfake detection systems struggle against unseen data. To address this, we employ three different deep Convolutional Neural Network (CNN) models, (1) VGG16, (2) InceptionV3, and (3) XceptionNet to classify fake and real images extracted from videos. We also constructed a fusion of the deep CNN models to improve the robustness and generalisation capability. The proposed technique outperforms state-of-the-art models with 96.5% accuracy, when tested on publicly available DeepFake Detection Challenge (DFDC) test data, comprising of 400 videos. The fusion model achieves 99% accuracy on lower quality DeepFake-TIMIT dataset videos and 91.88% on higher quality DeepFake-TIMIT videos. In addition to this, we prove that prediction fusion is more robust against adversarial attacks. If one model is compromised by an adversarial attack, the prediction fusion does not let it affect the overall classification.

Key findings

The fused prediction model achieved 96.5% accuracy on the DFDC test set, outperforming state-of-the-art models and achieving a better LogLoss score than the DFDC winning model. It also demonstrated high accuracy on DeepFake-TIMIT (99.68% on low-res and 91.88% on high-res videos), showing good generalization without specific training on this dataset. Furthermore, the prediction fusion proved to be robust against adversarial attacks, maintaining correct classification even when one constituent model was compromised.

Approach

The authors train three deep CNN models (VGG16, InceptionV3, XceptionNet) on augmented image frames extracted from videos to classify them as real or fake. For final prediction, they fuse the individual frame-by-frame predictions from these three CNNs by averaging them. This ensemble approach enhances the overall robustness and generalization capability, especially against adversarial attacks.

Datasets

DeepFake Detection Challenge (DFDC) dataset, DeepFake-TIMIT dataset (lower and higher resolution videos).

Model(s)

VGG16, InceptionV3, XceptionNet.

Author countries

Cyprus, UAE

← Previous