PRPO: Paragraph-level Policy Optimization for Vision-Language Deepfake Detection

Authors: Tuan Nguyen, Naseem Khan, Khang Tran, NhatHai Phan, Issa Khalil

Published: 2025-09-30 13:56:05+00:00

AI Summary

This paper introduces a reasoning-annotated dataset, DF-R5, and a novel reinforcement learning algorithm, Paragraph-level Relative Policy Optimization (PRPO), for vision-language deepfake detection. PRPO aligns multimodal large language model (LLM) reasoning with image content at the paragraph level, significantly improving detection accuracy and explanation faithfulness by grounding reasoning in visual evidence. This approach addresses the limitations of MLLMs in producing misaligned or hallucinatory deepfake explanations.

Abstract

The rapid rise of synthetic media has made deepfake detection a critical challenge for online safety and trust. Progress remains constrained by the scarcity of large, high-quality datasets. Although multimodal large language models (LLMs) exhibit strong reasoning capabilities, their performance on deepfake detection is poor, often producing explanations that are misaligned with visual evidence or hallucinatory. To address this limitation, we introduce a reasoning-annotated dataset for deepfake detection and propose Paragraph-level Relative Policy Optimization (PRPO), a reinforcement learning algorithm that aligns LLM reasoning with image content at the paragraph level. Experiments show that PRPO improves detection accuracy by a wide margin and achieves the highest reasoning score of 4.55/5.0. Ablation studies further demonstrate that PRPO significantly outperforms GRPO under test-time conditions. These results underscore the importance of grounding multimodal reasoning in visual evidence to enable more reliable and interpretable deepfake detection.

Key findings

PRPO significantly improves deepfake detection accuracy, achieving an 89.91% F1 score and outperforming state-of-the-art baselines and MLLMs. It also achieves a high reasoning score of 4.55/5.0, demonstrating enhanced explanation faithfulness and visual grounding. Ablation studies confirmed that combining Visual Consistency and Prediction Consistency Rewards in PRPO leads to balanced and superior performance.

Approach

The authors introduce DF-R5, a reasoning-annotated dataset, and propose DX-LLaVA, a multimodal architecture that integrates a CLIP ConvNeXT vision encoder with a Vicuna language model. They then develop Paragraph-level Relative Policy Optimization (PRPO), a test-time reinforcement learning algorithm that uses Visual Consistency Rewards (VCR) and Prediction Consistency Rewards (PCR) to align LLM reasoning with image content at a paragraph level, ensuring explanations are visually grounded and consistent with final predictions.

Datasets

DF-R5 (their introduced dataset), DF40 (which includes images from DDIM, PixArt-α, SD-2.1, SiT, StyleGAN3, FaceForensics++, CelebDF).

Model(s)

DX-LLaVA (their architecture, based on LLaVA), CLIP ConvNeXT (vision encoder), Vicuna (language model), PRPO (their reinforcement learning algorithm).

Author countries

Qatar, USA

← Previous