Metric-Aligned Group Relative Policy Optimization for Explainable Deepfake Detection
Authors: Luka Dragar, Peter Peer, Vitomir Štruc, Borut Batagelj
Published: 2026-11-10 18:08:35+00:00
Comment: 5 pages. Accepted to MRAC '26 (ACM Multimedia 2026)
AI Summary
This paper proposes a decoupled predict-then-explain pipeline for explainable deepfake detection, addressing the need for visual evidence explanations beyond simple real/fake classification. The approach utilizes a vision-foundation-model ensemble for initial classification, followed by a Qwen3-VL-8B model for decision-conditioned explanation generation. Metric-Aligned Group Relative Policy Optimization (GRPO) is applied to optimize explainers, resulting in improved BERTScore-F1 and grounding metrics.
Abstract
Deepfake detectors are commonly evaluated on real/fake classification alone, but real-world applications require explanations of the visual evidence behind each prediction. Joint vision-language models struggle with authenticity assessment, producing real narratives for manipulated images. To address this, we propose a decoupled predict-then-explain pipeline as our submission to the ACM Multimedia 2026 Explainable Deepfake Detection Challenge, with methods trained and evaluated on the XPlainVerse benchmark. A vision-foundation-model ensemble first classifies the image, then a Qwen3-VL-8B model generates a decision-conditioned explanation of the visual evidence. For fake images, a second Qwen3-VL-8B compresses that text into a short non-expert summary without using the image. We further apply metric-aligned Group Relative Policy Optimization (GRPO) to optimize both explainers against the challenge criteria. Under matched conditions, GRPO raises complex BERTScore-F1 from 0.686 to 0.712 and also improves grounding (entity and evidence F1); our submission ranked first on complex BERTScore-F1 and seventh overall.