Physics-Aware Deepfake Detection via Distance-Speech Consistency

Authors: Kyeongrae Kim, Kim Sung-Bin, Oh Hyun-Bin, Tae-Hyun Oh

Published: 2026-10-13 14:44:28+00:00

Comment: Interspeech 2026, Project Page: https://byulharang.github.io/PADD/

Journal Ref: Proc. Interspeech 2026, pp. 2424-2428 (2026)

AI Summary

This paper proposes a physics-aware deepfake detector for dynamic speaking videos that leverages the consistent relationship between speaker-to-camera distance and speech energy in real recordings. Unlike traditional lip-sync-based detectors which struggle with dynamic scenarios, this method combines video-based distance estimates with distance-dependent acoustic measures to identify deepfakes that break this physical coupling. Experiments show its effectiveness in dynamic settings and its ability to complement existing detectors through ensemble methods.

Abstract

Recent progress in generative models has made talking-head deepfakes highly realistic and easy to create, jeopardizing public trust. However, most audio-visual deepfake detectors rely on lip-speech synchronization and are designed for static, frontal videos, limiting their reliability in dynamic, in-the-wild recordings where the speaker moves and lip cues are degraded or missing. As an alternative, we propose a physics-aware detector for dynamic speaking videos that leverages an acoustic constraint: in real recordings, speech energy varies predictably with speaker-to-camera distance, whereas deepfake generation can break this coupling. Our method combines distance estimates from video with distance-dependent acoustic measures from speech to detect deepfakes. Experiments on in-the-wild benchmarks show that the proposed physical cue is effective in dynamic scenarios, and that it complements lip-sync-based detectors, with a simple ensemble achieving consistent gains across datasets. Codes are available at https://byulharang.github.io/PADD/.