PATE-Forensics: Perception-as-Tool for Explainable Deepfake Forensics with General-Purpose MLLMs

Authors: Yaqi Li, Jielun Peng, Yabin Wang, Jincheng Liu, Xiaopeng Hong

Published: 2026-08-19 06:10:00+00:00

Comment: 9 pages, 3 figures, 2 tables; DDL-X Track 3, IJCAI 2026 AI Safety Workshop

AI Summary

PATE-Forensics introduces a "Perception-as-Tool" paradigm for explainable deepfake forensics, decoupling detection and localization from explanation generation. It employs a DINOv3-based forensic perception tool for multi-granularity detection and cue-guided localization, whose outputs form structured context for a general-purpose MLLM to generate explanations without task-specific fine-tuning. This approach achieved the best official score on DDL-X Track 3, demonstrating its effectiveness.

Abstract

Existing explainable deepfake forensic methods typically rely on task-adapted MLLM to jointly address detection, localization, and explanation. Inspired by agent-style tool use, we instead introduce a Perception-as-Tool paradigm and instantiate it as PATE-Forensics, which architecturally decouples detection and localization from explanation generation while coupling detection and localization as tightly as possible within a forensic perception tool. The DINOv3-based tool couples a multi-granularity detection module that integrates global, patch-level, and segment-level evidence with a cue-guided localization module by spatializing the patch-level and segment-level evidence into forgery score maps that guide dense mask prediction. The original image and forensic perception outputs produced by the tool form structured forensic context for a general-purpose MLLM, which is guided by prompt constraints to generate explanations without task-specific fine-tuning. On DDL-X Track 3, PATE-Forensics achieves the best official score of 0.89, outperforming the second-ranked team by 0.19 points. Our code is available at https://github.com/yqli00000/PATE-Forensics.