AV-FABLE: Audio-Visual Manipulation Forensics in Long-Form Videos

Authors: Yuesheng Huang, Hoi Fan Au, Jiaqi Hou, Ruicheng Zhang, Jiarui He, Haocheng Yang, Bin Hu, Kewei Zhang, Weijian Deng, Jun Zhou, Xiu Li

Published: 2026-10-24 03:01:21+00:00

Comment: 41 pages, 10 figures, including appendices

AI Summary

This paper introduces AV-FABLE, a new dual-timeline benchmark for localizing sparse audio-visual manipulations in long-form videos, which uses separate audio and visual timelines to expose errors missed by joint-timeline evaluation. They find that zero-shot prompting of Audio-Visual Large Language Models (AV-LLMs) performs poorly, but supervised readouts of their frozen perception features recover strong forensic signals. Motivated by this, they propose NFSR (Native Forensics via Separated Readouts), which uses separate audio and visual temporal decoders on frozen AV-LLM features, achieving state-of-the-art localization across various settings.

Abstract

Localized AI manipulations in long-form videos often affect brief events within otherwise authentic content, making video-level classification insufficient for forensic analysis. Existing audio-visual deepfake benchmarks largely focus on short talking-head clips, while long-form temporal forensics remains predominantly visual. We introduce AV-FABLE, a dual-timeline benchmark for sparse manipulations in open-scene long-form audio-visual videos. AV-FABLE contains 16,650 pristine--manipulated twin pairs (998 hours) created with 27 editing configurations, with separate audio and visual timelines that enable modality-specific temporal localization and expose errors hidden by joint-timeline evaluation. On AV-FABLE, zero-shot prompting of audio-visual large language models (AV-LLMs) yields near-chance detection and poor temporal localization. In contrast, supervised readouts of their frozen perception features recover strong forensic signals, suggesting manipulation evidence is encoded in these representations but not effectively exposed through language outputs. Motivated by this observation, we propose NFSR(Native Forensics via Separated Readouts), which reads frozen AV-LLM features with separate audio and visual temporal decoders. NFSR achieves the best audio-visual localization performance in-domain and across all three out-of-distribution settings, including transfer to held-out videos from an external long-video forensic benchmark. AV-FABLE, code, and model weights will be publicly released.


Key findings
AV-FABLE reveals that zero-shot prompting of AV-LLMs performs poorly for manipulation detection and temporal localization, despite the underlying perception features containing strong forensic signals. NFSR, by utilizing separate audio and visual temporal decoders on frozen AV-LLM features, achieves superior audio-visual localization performance in-domain and across various out-of-distribution settings. The study highlights the importance of modality-separated temporal readouts for effective long-form audio-visual forensics.
Approach
The authors introduce AV-FABLE, a benchmark of 16,650 pristine-manipulated twin pairs of long-form audio-visual videos with dual timelines for sparse manipulations. They then propose NFSR, a method that reads frozen features from pre-trained Audio-Visual Large Language Models (AV-LLMs) using separate audio and visual temporal decoders, conditioned on the host video's statistics, to localize manipulations on their respective timelines.
Datasets
AV-FABLE, TASLE, HDTF, ActivityNet, EPIC-KITCHENS-100
Model(s)
Qwen2.5-Omni, Qwen3-Omni, video-SALMONN 2+, Xception, AASIST, DeMamba, BA-TFD+, ActionFormer, UMMAFormer, AuViRe, ICS-Sync, DAV-Det, Whisper-large-v3, BEATs, CLIP
Author countries
UNKNOWN