Fit for Purpose? Deepfake Detection in the Real World

View on arXiv ← Back to list

Authors: Guangyu Lin, Li Lin, Christina P. Walker, Daniel S. Schiff, Shu Hu

Published: 2025-10-18 16:00:10+00:00

AI Summary

This work introduces the Political Deepfakes Incident Database (PDID), a curated collection of real-world political deepfakes shared on social platforms, to serve as a systematic benchmark for deepfake detection. A comprehensive evaluation of state-of-the-art detectors from academia, government, and industry reveals that most models struggle significantly to generalize effectively to these authentic political deepfakes, and often fail under real-world conditions. The results emphasize the urgent need for politically contextualized and robust deepfake detection frameworks to safeguard the public interest.

Abstract

The rapid proliferation of AI-generated content, driven by advances in generative adversarial networks, diffusion models, and multimodal large language models, has made the creation and dissemination of synthetic media effortless, heightening the risks of misinformation, particularly political deepfakes that distort truth and undermine trust in political institutions. In turn, governments, research institutions, and industry have strongly promoted deepfake detection initiatives as solutions. Yet, most existing models are trained and validated on synthetic, laboratory-controlled datasets, limiting their generalizability to the kinds of real-world political deepfakes circulating on social platforms that affect the public. In this work, we introduce the first systematic benchmark based on the Political Deepfakes Incident Database, a curated collection of real-world political deepfakes shared on social media since 2018. Our study includes a systematic evaluation of state-of-the-art deepfake detectors across academia, government, and industry. We find that the detectors from academia and government perform relatively poorly. While paid detection tools achieve relatively higher performance than free-access models, all evaluated detectors struggle to generalize effectively to authentic political deepfakes, and are vulnerable to simple manipulations, especially in the video domain. Results urge the need for politically contextualized deepfake detection frameworks to better safeguard the public in real-world settings.

Key findings

Academic and government deepfake detectors perform poorly on real-world political deepfakes, achieving a maximum AUC below 75%, suggesting they are unsuitable for public use without extensive fine-tuning. While paid commercial tools and paid LVLMs achieve higher discriminative performance (AUC), they often exhibit high False Acceptance Rates (FARs), requiring expert threshold calibration. Political video deepfake detection remains a significant, persistent challenge, with most detectors showing a notable drop in performance compared to image detection results.

Approach

The authors introduce the Political Deepfakes Incident Database (PDID) comprising verified real-world political deepfakes (images and videos) disseminated since 2018. They categorize existing detection systems into LVLM-agnostic (white-box from academia/government and black-box commercial) and LVLM-aware detectors (Large Vision-Language Models). The performance of these detectors is systematically evaluated on the PDID dataset using metrics like ACC, AUC, and FAR, along with robustness assessments against common post-processing operations.

Datasets

Political Deepfakes Incident Database (PDID). Also referenced: FaceForensics++, Celeb-DF, DFDC, AI-Face, FFHQ, IMDB-WIKI.

Model(s)

Xception, EfficientNet-B4, ViT-B/16, F3Net, SRM, UCF, PG-FDD, CNNDetection, GANattribution (representing white-box models); Commercial tools (e.g., Reality Defender, Hive Moderation); Large Vision-Language Models (LVLMs) including ChatGPT (GPT-5), Claude-Sonnet-4.5, and LLaVA variants.

Author countries

USA

← Previous