Towards Solving the DeepFake Problem : An Analysis on Improving DeepFake Detection using Dynamic Face Augmentation

Authors: Sowmen Das, Selim Seferbekov, Arup Datta, Md. Saiful Islam, Md. Ruhul Amin

Published: 2021-02-18 20:25:45+00:00

AI Summary

This paper addresses the problem of deepfake detection models overfitting due to highly oversampled datasets, causing them to memorize faces rather than fake features. The authors propose Face-Cutout, a dynamic data augmentation method that uses facial landmark information to selectively occlude image regions. This technique successfully improves data variation, alleviates overfitting, and enhances the generalizability of deepfake detection models.

Abstract

The creation of altered and manipulated faces has become more common due to the improvement of DeepFake generation methods. Simultaneously, we have seen detection models' development for differentiating between a manipulated and original face from image or video content. In this paper, we focus on identifying the limitations and shortcomings of existing deepfake detection frameworks. We identified some key problems surrounding deepfake detection through quantitative and qualitative analysis of existing methods and datasets. We found that deepfake datasets are highly oversampled, causing models to become easily overfitted. The datasets are created using a small set of real faces to generate multiple fake samples. When trained on these datasets, models tend to memorize the actors' faces and labels instead of learning fake features. To mitigate this problem, we propose a simple data augmentation method termed Face-Cutout. Our method dynamically cuts out regions of an image using the face landmark information. It helps the model selectively attend to only the relevant regions of the input. Our evaluation experiments show that Face-Cutout can successfully improve the data variation and alleviate the problem of overfitting. Our method achieves a reduction in LogLoss of 15.2% to 35.3% on different datasets, compared to other occlusion-based techniques. Moreover, we also propose a general-purpose data pre-processing guideline to train and evaluate existing architectures allowing us to improve the generalizability of these models for deepfake detection.

Key findings

Face-Cutout significantly improves deepfake detection performance by reducing overfitting and improving data variation. The method achieves a reduction in LogLoss of 15.2% to 35.3% across different datasets compared to other occlusion-based techniques. It also enhances the generalizability and robustness of existing vision architectures for deepfake detection.

Approach

The authors propose Face-Cutout, a data augmentation method that dynamically cuts out regions of an image based on facial landmark information to improve data variation. This helps the model selectively attend to relevant regions and prevents it from overfitting by memorizing specific faces. They also introduce a pre-processing guideline using face clustering to prevent data leaks during dataset splitting.

Datasets

DFDC, FaceForensics++ (FF++), Celeb-DF

Model(s)

UNKNOWN

Author countries

Bangladesh, Canada, USA

← Previous