Exploring the Impact of Moire Pattern on Deepfake Detectors

Authors: Razaib Tariq, Shahroz Tariq, Simon S. Woo

Published: 2024-07-15 02:39:24+00:00

Comment: 7 page, 4 figures, 1 table, Accepted for publication in IEEE International Conference on Image Processing (ICIP 2024)

AI Summary

This study investigates the previously unexplored impact of Moiré patterns on deepfake detector performance, specifically when deepfake videos are captured by cameras from digital screens. Experiments on CelebDF and FF++ datasets with four state-of-the-art detectors revealed a significant decline in accuracy, with none surpassing 68% on average. This underscores a critical vulnerability in real-world deepfake detection scenarios, necessitating solutions to address Moiré pattern challenges.

Abstract

Deepfake detection is critical in mitigating the societal threats posed by manipulated videos. While various algorithms have been developed for this purpose, challenges arise when detectors operate externally, such as on smartphones, when users take a photo of deepfake images and upload on the Internet. One significant challenge in such scenarios is the presence of Moiré patterns, which degrade image quality and confound conventional classification algorithms, including deep neural networks (DNNs). The impact of Moiré patterns remains largely unexplored for deepfake detectors. In this study, we investigate how camera-captured deepfake videos from digital screens affect detector performance. We conducted experiments using two prominent datasets, CelebDF and FF++, comparing the performance of four state-of-the-art detectors on camera-captured deepfake videos with introduced Moiré patterns. Our findings reveal a significant decline in detector accuracy, with none achieving above 68% on average. This underscores the critical need to address Moiré pattern challenges in real-world deepfake detection scenarios.

Key findings

The study found a significant decline in the performance of all evaluated state-of-the-art deepfake detectors when exposed to camera-captured deepfake videos containing Moiré patterns, with average accuracy not exceeding 68%. Even the best-performing detector, ADD, could not surpass 68% accuracy, indicating a substantial performance degradation compared to typical high 90s accuracy on clean data. This reveals a critical vulnerability in current deepfake detection systems against this type of real-world artifact and highlights the urgent need for robust mitigation techniques.

Approach

The authors simulate real-world deepfake detection scenarios by displaying deepfake videos from CelebDF and FF++ datasets on a computer screen and re-recording them using a smartphone camera, which inherently introduces Moiré patterns. They then evaluate the performance (F1-score, AUROC) of several state-of-the-art deepfake detectors on these camera-captured videos to assess the impact of Moiré patterns.

Datasets

CelebDF, Faceforensics++ (FF++)

Model(s)

ADD, Xception, CLRNet, QAD, CoReD, BZNet

Author countries

South Korea, Australia

← Previous