The Deepfake Detection Challenge (DFDC) Preview Dataset

Authors: Brian Dolhansky, Russ Howes, Ben Pflaum, Nicole Baram, Cristian Canton Ferrer

Published: 2019-10-19 22:35:52+00:00

AI Summary

This paper introduces a preview of the Deepfakes Detection Challenge (DFDC) dataset, comprising 5K videos with facial manipulations generated by two algorithms. The dataset emphasizes diversity in actors and backgrounds, with explicit consent for likeness manipulation. The authors also define specific evaluation metrics and provide baseline performance results using existing deepfake detection models.

Abstract

In this paper, we introduce a preview of the Deepfakes Detection Challenge (DFDC) dataset consisting of 5K videos featuring two facial modification algorithms. A data collection campaign has been carried out where participating actors have entered into an agreement to the use and manipulation of their likenesses in our creation of the dataset. Diversity in several axes (gender, skin-tone, age, etc.) has been considered and actors recorded videos with arbitrary backgrounds thus bringing visual variability. Finally, a set of specific metrics to evaluate the performance have been defined and two existing models for detecting deepfakes have been tested to provide a reference performance baseline. The DFDC dataset preview can be downloaded at: deepfakedetectionchallenge.ai

Key findings

The baseline models, TamperNet and XceptionNet, demonstrated varying performance on the DFDC preview dataset. XceptionNet (Face) generally achieved the highest log-weighted precision, while XceptionNet (Full) showed higher recall at the cost of lower precision. The results highlighted the challenge of effectively detecting deepfakes, especially when considering the proposed weighted precision metric that accounts for low prevalence.

Approach

The authors collected a diverse set of videos from crowdsourced actors who consented to their likeness being manipulated. They applied two facial modification algorithms to create deepfakes and defined new evaluation metrics (weighted precision and recall) to address the low prevalence of deepfakes in real-world traffic. Baseline performance was established using two existing deepfake detection models.

Datasets

DFDC Preview Dataset, FaceForensics

Model(s)

TamperNet (a small DNN), XceptionNet (Face and Full-image variants)

Author countries

UNKNOWN

← Previous