Exploring Strengths and Weaknesses of Super-Resolution Attack in Deepfake Detection

Authors: Davide Alessandro Coccomini, Roberto Caldelli, Fabrizio Falchi, Claudio Gennaro, Giuseppe Amato

Published: 2024-10-05 15:47:34+00:00

Comment: Trust What You learN (TWYN) Workshop at European Conference on Computer Vision ECCV 2024

AI Summary

This paper investigates the effectiveness of super-resolution (SR) techniques as an adversarial attack to deceive deepfake detectors. It explores different SR methods and scales, demonstrating that these attacks effectively camouflage artifacts in deepfakes derived from real images but fail against fully synthetic content. The research also proposes and evaluates data augmentation strategies to enhance deepfake detectors' robustness against SR-based attacks.

Abstract

Image manipulation is rapidly evolving, allowing the creation of credible content that can be used to bend reality. Although the results of deepfake detectors are promising, deepfakes can be made even more complicated to detect through adversarial attacks. They aim to further manipulate the image to camouflage deepfakes' artifacts or to insert signals making the image appear pristine. In this paper, we further explore the potential of super-resolution attacks based on different super-resolution techniques and with different scales that can impact the performance of deepfake detectors with more or less intensity. We also evaluated the impact of the attack on more diverse datasets discovering that the super-resolution process is effective in hiding the artifacts introduced by deepfake generation models but fails in hiding the traces contained in fully synthetic images. Finally, we propose some changes to the detectors' training process to improve their robustness to this kind of attack.

Key findings

Super-resolution (SR) attacks significantly degrade deepfake detector performance when applied to manipulated real images, with BSRGAN being more effective than EDSR. The attack's potency increases with higher scaling factors, but this also introduces more visual distortion. Crucially, the SR attack is ineffective against deepfake detectors classifying entirely synthetic images, indicating a difference in underlying artifacts. Training deepfake detectors with SR-based data augmentation improves their robustness against these attacks.

Approach

The authors propose a black-box adversarial attack where faces within deepfake video frames are extracted, downscaled, and then super-resolved using techniques like EDSR or BSRGAN before being reinserted. This process aims to blur deepfake artifacts. They evaluate this attack against various deepfake detectors and suggest incorporating SR-based data augmentation during training to improve detector resilience.

Datasets

FaceForensics++ (FF++), dataset of synthetic images generated with StyleGAN, StyleGAN2, RelGAN, and ProGAN.

Model(s)

Resnet50, Swin-Small, XceptionNet (for deepfake detection); EDSR, BSRGAN (for super-resolution); MTCNN (for face detection).

Author countries

Italy

← Previous