Exposing DeepFakes via Hyperspectral Domain Mapping

View on arXiv ← Back to list

Authors: Aditya Mehta, Swarnim Chaudhary, Pratik Narang, Jagat Sesh Challa

Published: 2025-11-13 06:25:44+00:00

AI Summary

The paper introduces HSI-Detect, a two-stage pipeline for deepfake detection that reconstructs a 31-channel hyperspectral image (HSI) from standard RGB inputs. Detection is then performed in this expanded spectral domain, which amplifies manipulation artifacts often weak or invisible in RGB space. The method achieves consistent improvements over RGB-only baselines on the FaceForensics++ dataset.

Abstract

Modern generative and diffusion models produce highly realistic images that can mislead human perception and even sophisticated automated detection systems. Most detection methods operate in RGB space and thus analyze only three spectral channels. We propose HSI-Detect, a two-stage pipeline that reconstructs a 31-channel hyperspectral image from a standard RGB input and performs detection in the hyperspectral domain. Expanding the input representation into denser spectral bands amplifies manipulation artifacts that are often weak or invisible in the RGB domain, particularly in specific frequency bands. We evaluate HSI-Detect across FaceForensics++ dataset and show the consistent improvements over RGB-only baselines, illustrating the promise of spectral-domain mapping for Deepfake detection.

Key findings

HSI-Detect consistently outperformed RGB-only baselines (ViT, RECCE, MoE-FFD) across multiple unseen manipulation types in cross-manipulation detection. It achieved the highest overall average AUC of 68.92, demonstrating the advantage of hyperspectral features over standard RGB features, particularly for DeepFakes and FaceSwap manipulations.

Approach

HSI-Detect first uses the MST++ model to reconstruct a 31-channel hyperspectral image from the 3-channel RGB input. Subsequently, an enhanced Spectral Detection Network based on UCF, which uses a Disentanglement Framework, analyzes these dense spectral bands to identify subtle forgery artifacts.

Datasets

FaceForensics++ (trained on Neural Textures sub-dataset, evaluated cross-manipulation)

Model(s)

MST++, Enhanced UCF (using a Disentanglement Framework)

Author countries

India

← Previous