Deep Learning for Deepfakes Creation and Detection: A Survey

View on arXiv ← Back to list

Authors: Thanh Thi Nguyen, Quoc Viet Hung Nguyen, Dung Tien Nguyen, Duc Thanh Nguyen, Thien Huynh-The, Saeid Nahavandi, Thanh Tam Nguyen, Quoc-Viet Pham, Cuong M. Nguyen

Published: 2019-09-25 16:03:45+00:00

Journal Ref: Computer Vision and Image Understanding, 223 (2022) 103525

AI Summary

This survey paper comprehensively reviews deep learning algorithms for both the creation and detection of deepfakes. It categorizes existing detection methods based on data type (image or video) and features used, highlighting their current capabilities and limitations. The study aims to provide a structured overview of deepfake technologies and inspire the development of more robust countermeasures against emerging threats.

Abstract

Deep learning has been successfully applied to solve various complex problems ranging from big data analytics to computer vision and human-level control. Deep learning advances however have also been employed to create software that can cause threats to privacy, democracy and national security. One of those deep learning-powered applications recently emerged is deepfake. Deepfake algorithms can create fake images and videos that humans cannot distinguish them from authentic ones. The proposal of technologies that can automatically detect and assess the integrity of digital visual media is therefore indispensable. This paper presents a survey of algorithms used to create deepfakes and, more importantly, methods proposed to detect deepfakes in the literature to date. We present extensive discussions on challenges, research trends and directions related to deepfake technologies. By reviewing the background of deepfakes and state-of-the-art deepfake detection methods, this study provides a comprehensive overview of deepfake techniques and facilitates the development of new and more robust methods to deal with the increasingly challenging deepfakes.

Key findings

Deepfakes are becoming increasingly realistic and easier to create, posing significant threats to privacy, democracy, and national security. Existing deep learning-based detection methods often struggle with generalization across different deepfake types and datasets, and are vulnerable to adversarial attacks. Future research needs to focus on developing more robust, scalable, and generalizable detection techniques, integrating them into distribution platforms, utilizing blockchain for digital provenance, and improving explainable AI for forensic applications.

Approach

The paper surveys deep learning algorithms for deepfake creation and detection, categorizing detection methods based on data type (fake images vs. fake videos) and features (handcrafted vs. deep, or temporal vs. visual artifacts). It extensively discusses challenges, research trends, and future directions in deepfake technologies.

Datasets

FaceForensics++, DeepfakeDetection, Celeb-DF (v1 & v2), Deepfake Detection Challenge (DFDC), DFDC Preview, DeeperForensics-1.0, UADFV, DeepfakeTIMIT, CelebA/CelebA-HQ, ILSVRC12, VidTIMIT, ASVSpoof 2019, COHFACE, Labeled Faces in the Wild (LFW), ForenSynths, Idiap Research Institute replay-attack dataset, Hollywood human actions dataset, and the World Leaders dataset. Various GAN-generated datasets (e.g., DCGAN, WGAN, PGGAN, StyleGAN) are also mentioned as sources of fake data.

Model(s)

For audio deepfake detection or joint audio-video models: Convolution modules (for audio embeddings), Siamese network (for audio-visual affective cues). The paper also extensively covers models for visual deepfake detection, including various CNN architectures (DenseNet, ResNet, VGG, XceptionNet, MesoNet, SCnet), Recurrent Neural Networks (LSTM, GRU, RCN, LRCN), Capsule Networks, and shallow classifiers like SVM, Random Forest, MLP, Logistic Regression, KNN, and LDA.

Author countries

Australia, Republic of Korea, Vietnam, France

← Previous