Scalable, Energy-Efficient Optical-Neural Architecture for Multiplexed Deepfake Video Detection

Authors: Parnian Ghapandar Kashani, Shiqi Chen, Aydogan Ozcan

Published: 2026-05-19 04:42:47+00:00

Comment: 30 Pages, 8 Figures

AI Summary

This paper presents a hybrid digital-analog framework for deepfake video detection, combining a lightweight digital front-end with a spatially multiplexed optical decoding back-end. This approach enables massively parallel analog inference, allowing simultaneous processing of 15 or more video streams in a single optical pass. The system achieves high-throughput, accurate video-level authenticity prediction with reduced computational cost and demonstrates resilience against various degradations and adversarial attacks.

Abstract

The rapid proliferation of AI-generated visual media has created an urgent need for efficient, trustworthy deepfake detection systems. However, existing deep learning-based detection methods rely on computationally intensive and energy-demanding inference algorithms, limiting their scalability. Here, we present a hybrid digital-analog deepfake video detection framework that combines a lightweight digital front-end with a spatially multiplexed optical decoding back-end for massively parallel analog inference through a programmable spatial light modulator. By simultaneously processing 15 or more video streams within a single optical propagation pass, the system enables high-throughput and accurate video-level authenticity prediction at reduced computational cost compared with purely digital methods. We validated this hybrid deepfake video processor using different datasets spanning classical face-swapping, real-world deepfake recordings, and fully AI-generated videos. Using a spatially multiplexed experimental set-up operating in the visible spectrum, we achieved average deepfake detection accuracy, sensitivity and specificity of 97.79%, 99.86% and 95.72%, respectively, on the Celeb-DF video dataset with 15 videos tested in parallel in a single optical pass per inference. The multiplexed optical decoder also demonstrates resilience against various types of video degradation, noise, compression, experimental misalignments and black-box adversarial attacks. Our results show that integrating optical computation into AI inference enables simultaneous gains in throughput, energy efficiency, and adversarial robustness - three properties that are difficult to achieve together in purely digital systems.

Key findings

The hybrid system achieved an average deepfake detection accuracy of 97.79%, sensitivity of 99.86%, and specificity of 95.72% on the Celeb-DF dataset. It demonstrated high throughput and significant energy efficiency compared to purely digital methods. Furthermore, the system exhibited robust performance against video degradation, noise, compression, physical misalignments, and black-box adversarial attacks, highlighting the benefits of integrating optical computation into AI inference for improved throughput, energy efficiency, and adversarial robustness.

Approach

The approach utilizes a hybrid digital-optical architecture. A lightweight digital encoder first processes video frames using spatial and Fourier-domain CNNs to generate a 2D phase pattern. This pattern is then loaded onto a spatial light modulator (SLM) for a spatially multiplexed optical decoder, which performs parallel analog inference through light propagation and optimized diffractive layers to classify videos.

Datasets

Celeb-DF (V2), Google VEO-3 (Gemini), DeepSpeak

Model(s)

UNKNOWN

Author countries

USA

← Previous