Metric-Aligned Group Relative Policy Optimization for Explainable Deepfake Detection

Authors: Luka Dragar, Peter Peer, Vitomir Štruc, Borut Batagelj

Published: 2026-11-10 18:08:35+00:00

Comment: 5 pages. Accepted to MRAC '26 (ACM Multimedia 2026)

AI Summary

This paper proposes a decoupled predict-then-explain pipeline for explainable deepfake detection, addressing the need for visual evidence explanations beyond simple real/fake classification. The approach utilizes a vision-foundation-model ensemble for initial classification, followed by a Qwen3-VL-8B model for decision-conditioned explanation generation. Metric-Aligned Group Relative Policy Optimization (GRPO) is applied to optimize explainers, resulting in improved BERTScore-F1 and grounding metrics.

Abstract

Deepfake detectors are commonly evaluated on real/fake classification alone, but real-world applications require explanations of the visual evidence behind each prediction. Joint vision-language models struggle with authenticity assessment, producing real narratives for manipulated images. To address this, we propose a decoupled predict-then-explain pipeline as our submission to the ACM Multimedia 2026 Explainable Deepfake Detection Challenge, with methods trained and evaluated on the XPlainVerse benchmark. A vision-foundation-model ensemble first classifies the image, then a Qwen3-VL-8B model generates a decision-conditioned explanation of the visual evidence. For fake images, a second Qwen3-VL-8B compresses that text into a short non-expert summary without using the image. We further apply metric-aligned Group Relative Policy Optimization (GRPO) to optimize both explainers against the challenge criteria. Under matched conditions, GRPO raises complex BERTScore-F1 from 0.686 to 0.712 and also improves grounding (entity and evidence F1); our submission ranked first on complex BERTScore-F1 and seventh overall.


Key findings
The decoupled predict-then-explain pipeline effectively addresses the decision bias observed in joint vision-language models, improving explanation quality. Applying metric-aligned Group Relative Policy Optimization (GRPO) significantly enhances explanation metrics, raising complex BERTScore-F1 from 0.686 to 0.712 and improving grounding (entity and evidence F1). The submission ranked first in complex BERTScore-F1 and seventh overall in the ACM Multimedia 2026 Explainable Deepfake Detection Challenge, with detection F1 being the primary limitation rather than explanation quality.
Approach
The proposed method employs a decoupled predict-then-explain pipeline. First, a vision-foundation-model ensemble (SigLIP2-SO400M + DINOv2-Large and DINOv3-MAC) classifies the image as real or fake. Then, a Qwen3-VL-8B model generates a decision-conditioned explanation of the visual evidence. For fake images, a second Qwen3-VL-8B compresses this text into a short non-expert summary. The explanation generation process is further optimized using metric-aligned Group Relative Policy Optimization (GRPO).
Datasets
XPlainVerse, OpenFake, DFBench, SID-Set, NTIRE 2026 Robust AI-Generated Image Detection training set
Model(s)
SigLIP2-SO400M, DINOv2-Large, DINOv3-MAC, Qwen3-VL-8B-Instruct
Author countries
Slovenia