The DeepFake Detection Challenge (DFDC) Dataset

Authors: Brian Dolhansky, Joanna Bitton, Ben Pflaum, Jikuo Lu, Russ Howes, Menglin Wang, Cristian Canton Ferrer

Published: 2020-06-12 18:15:55+00:00

AI Summary

The paper introduces the DeepFake Detection Challenge (DFDC) dataset, the largest publicly available face swap video dataset, comprising over 100,000 clips from 3,426 paid actors generated using various Deepfake methods. It details the dataset's construction and provides an analysis of top submissions from the accompanying Kaggle competition. The study concludes that models trained on the DFDC dataset can generalize to real-world Deepfake videos, despite the detection problem remaining extremely difficult.

Abstract

Deepfakes are a recent off-the-shelf manipulation technique that allows anyone to swap two identities in a single video. In addition to Deepfakes, a variety of GAN-based face swapping methods have also been published with accompanying code. To counter this emerging threat, we have constructed an extremely large face swap video dataset to enable the training of detection models, and organized the accompanying DeepFake Detection Challenge (DFDC) Kaggle competition. Importantly, all recorded subjects agreed to participate in and have their likenesses modified during the construction of the face-swapped dataset. The DFDC dataset is by far the largest currently and publicly available face swap video dataset, with over 100,000 total clips sourced from 3,426 paid actors, produced with several Deepfake, GAN-based, and non-learned methods. In addition to describing the methods used to construct the dataset, we provide a detailed analysis of the top submissions from the Kaggle contest. We show although Deepfake detection is extremely difficult and still an unsolved problem, a Deepfake detection model trained only on the DFDC can generalize to real in-the-wild Deepfake videos, and such a model can be a valuable analysis tool when analyzing potentially Deepfaked videos. Training, validation and testing corpuses can be downloaded from https://ai.facebook.com/datasets/dfdc.

Key findings

Deepfake detection remains a challenging and largely unsolved problem. However, models trained exclusively on the DFDC dataset demonstrate the ability to generalize and detect real 'in-the-wild' Deepfake videos, making the dataset a valuable resource for training robust detection systems. Top-performing models from the challenge, often ensembles of architectures like EfficientNet and Xception, achieved an average precision of 0.753 and a ROC-AUC score of 0.734 on real Deepfakes.

Approach

The authors address the Deepfake detection problem by constructing the DeepFake Detection Challenge (DFDC) dataset, an extensive collection of over 100,000 face-swapped video clips, and organizing a public Kaggle competition. They describe the dataset's generation methods (DFAE, MM/NN, NTH, FSGAN, StyleGAN) and analyze the performance of top-contributing models submitted by participants.

Datasets

DeepFake Detection Challenge (DFDC) Dataset

Model(s)

UNKNOWN

Author countries

United States

← Previous