GRIDEX: Grid-Grounded Forensic Explanations for Deepfake Spectrogram Analysis

Authors: Thi Ngan Ha Do, Tingmin Wu, Alsharif Abuadbba, Kristen Moore

Published: 2026-06-17 06:32:15+00:00

AI Summary

This paper introduces GRIDEX, a novel pipeline for generating grid-grounded forensic explanations for deepfake spectrograms. Unlike existing deepfake detection models that only provide classification, GRIDEX localizes anomalous regions in a spectrogram and provides structured, auditable explanations for each anomaly, detailing its temporal, spectral, and phonetic context. It is trained using a two-stage learning paradigm combining supervised fine-tuning (SFT) with Group Relative Policy Optimization (GRPO) to improve artifact localization and explanation quality.

Abstract

The advancement of speech generation technologies has made artificial speech increasingly realistic. Although modern classification models can achieve high accuracy when it comes to deepfake detection, they do not produce evidences such as indicating where spoof cues appear in the spectrogram and what they imply acoustically, limiting their usefulness in forensic settings. Manual analysis of full spectrograms is resource-intensive, so evidence should narrow attention to the most diagnostic regions. Moreover, existing explainability methods have limited capabilities in connecting contextual attributes to localized evidence, making explanations harder to verify. To overcome this limitation, we propose GRIDEX, a pipeline that, when given a deepfake spectrogram, generates forensic explanations of its anomalies. The pipeline (i) selects top-K anomalous regions in the spectrogram and (ii) produces an explanation for each anomaly. The explanations follow a schema of categorical acoustic fields, including temporal, spectral, phonetic information and interpretation text. To our knowledge, this is the first framework to generate structured forensic explanations using regional grounding for deepfake spectrograms. GRIDEX is trained with a two-stage learning paradigm that combines supervised fine-tuning (SFT) with Group Relative Policy Optimization (GRPO). Experiments on our dataset show improved artifact localization and explanation quality over strong vision-language model (VLM) baselines. The dataset and code will be released upon publication.

Key findings

GRIDEX significantly outperforms strong VLM baselines in artifact localization (e.g., R@3: 0.386 vs. 0.241, nDCG: 0.411 vs. 0.244). The staged SFT and GRPO training strategy proves effective, substantially improving both localization accuracy and structured explanation grounding (CovAvg: 0.884 vs. 0.643 for best baseline). The system provides verifiable, region-grounded explanations aligning with common acoustic artifact categories, enabling more robust forensic analysis.

Approach

GRIDEX operates in two sequential stages: first, it identifies the top-K anomalous regions within a fixed GxG grid overlayed on a deepfake spectrogram (Query 1); second, for each selected region, it generates a structured explanation tuple containing categorical acoustic fields (temporal, spectral, phonetic) and an interpretation text (Query 2). This process utilizes a Vision-Language Model (VLM) backbone with turn-conditioned parameter-efficient fine-tuning (PEFT) and reinforcement learning (GRPO) for optimization.

Datasets

VocV4 corpus (to construct a new region-grounded explanation dataset from it)

Model(s)

Qwen2.5-VL-3B-Instruct (backbone with LoRA PEFT), VLM baselines (LLaVA-OneVision-1.5-8B-Instruct, Qwen2.5-VL-32B-Instruct, Qwen3-VL-8B-Instruct, InternVL3-78B)

Author countries

Australia

← Previous