Media Forensics and DeepFakes: an overview

Authors: Luisa Verdoliva

Published: 2020-01-18 00:13:32+00:00

AI Summary

This review paper provides a comprehensive analysis of methods for visual media integrity verification, with a particular focus on the emerging phenomenon of deepfakes and modern data-driven forensic techniques. It examines both conventional and deep learning-based approaches, highlighting their current limitations, challenges, and suggesting future research directions in the rapidly evolving field of media forensics. The paper emphasizes the need for robust automated tools to detect manipulated images and videos amidst growing security threats posed by realistic synthetic media.

Abstract

With the rapid progress of recent years, techniques that generate and manipulate multimedia content can now guarantee a very advanced level of realism. The boundary between real and synthetic media has become very thin. On the one hand, this opens the door to a series of exciting applications in different fields such as creative arts, advertising, film production, video games. On the other hand, it poses enormous security threats. Software packages freely available on the web allow any individual, without special skills, to create very realistic fake images and videos. So-called deepfakes can be used to manipulate public opinion during elections, commit fraud, discredit or blackmail people. Potential abuses are limited only by human imagination. Therefore, there is an urgent need for automated tools capable of detecting false multimedia content and avoiding the spread of dangerous false information. This review paper aims to present an analysis of the methods for visual media integrity verification, that is, the detection of manipulated images and videos. Special emphasis will be placed on the emerging phenomenon of deepfakes and, from the point of view of the forensic analyst, on modern data-driven forensic methods. The analysis will help to highlight the limits of current forensic tools, the most relevant issues, the upcoming challenges, and suggest future directions for research.

Key findings

Deep learning methods achieve high detection accuracy in ideal, aligned training and test conditions but suffer signiﬁcant performance drops when exposed to unseen manipulations or real-world post-processing like strong compression. One-class methods that focus on anomalies from pristine data, and fusion strategies combining multiple forensic tools, show promise for improved generalization and robustness against diverse and evolving attacks. The field faces an ongoing 'arms race,' necessitating more adaptable, robust, and interpretable deep learning solutions, alongside multi-asset and semantic-level analyses.

Approach

This review paper solves the problem by systematically analyzing and categorizing existing research in visual media forensics, covering conventional techniques, deep learning-based methods, and specialized deepfake detection. It dissects how various approaches leverage forensic traces, such as compression artifacts, noise patterns, and statistical anomalies, to identify manipulated images and videos. The authors also discuss the datasets, models, and challenges, concluding with perspectives on future research directions and the inherent 'arms race' between content manipulators and detectors.

Datasets

Casia v2, NIST MFC2019, DEFACTO, FaceForensics++, Celeb-DF, DFDC, DeeperForensics-1.0

Model(s)

UNKNOWN

Author countries

Italy

← Previous