Pulse or Provenance? What rPPG-Based Deepfake Detectors Learn

Authors: Othmane Harraq, Tamer Aldwairi

Published: 2026-11-10 12:36:56+00:00

AI Summary

This paper investigates what rPPG-based deepfake detectors truly learn, finding that they do not primarily detect physiological cardiac signals. Instead, these detectors leverage high-frequency temporal color artifacts introduced by image-to-video synthesis and encode video source provenance, leading to misclassifications on real videos from different sources.

Abstract

Remote photoplethysmography (rPPG) has been proposed as a physiological cue for deepfake detection, based on the premise that synthesized faces lack a genuine cardiac signal. We examine what rPPG-style detectors learn on Celeb-DF++ (22 forgery methods) and FaceForensics++ (FF++), using one identically sampled window per real and fake video. On talking-face forgeries, simple chrominance traces reach an AUC of 0.998, exceeding a dedicated rPPG extractor (0.87). Controlled experiments indicate that this signal is not physiological in origin. On fully synthesized forgeries, it is equally discriminative when extracted from background regions or from full frames without face detection, and it is concentrated above the 0.7 to 4 Hz heart-rate band. It is largely removed by shuffling frame order but persists after normalizing resolution, frame rate and encoding, and an independent reimplementation reproduces these findings. Removing high-frequency content makes most synthesized videos appear real, adding it makes real videos appear fake, and injecting a synthetic pulse has little effect. In controlled self-reenactment experiments with two generators, animating a face from a single frame introduces the same high-frequency signal, whereas reconstructing each frame from its own source does not. Face-based chrominance detectors trained on Celeb-DF++ classify over 99% of real FF++ videos as fake, indicating that they largely encode video source rather than manipulation, and a CLIP-based detector exhibits the same behavior. A frozen CLIP backbone matches or exceeds fusions of CLIP with rPPG and frequency features. Finally, we show that the official Celeb-DF and Celeb-DF++ splits share identities and source-video content between training and test, and that custom protocols reusing test videos and identities in training inflate weaker detectors by up to 0.30 AUC.


Key findings
rPPG-style detectors primarily detect high-frequency temporal color artifacts (above the heart-rate band) introduced during image-to-video synthesis, rather than physiological signals. These detectors largely encode video source, classifying real videos from different datasets as fake. Furthermore, common evaluation practices using overlapping training and test data in datasets like Celeb-DF++ inflate reported AUC scores, particularly for weaker detectors.
Approach
The authors analyze chrominance traces and rPPG extractors on various deepfake datasets, controlling for factors like sampling, region of interest (face, background, full-frame), frequency bands, and normalization. They conduct self-reenactment experiments and cross-dataset evaluations to understand the origin and nature of the detected signals, also examining the impact of dataset splits and evaluation protocols.
Datasets
Celeb-DF++, FaceForensics++ (FF++), Celeb-DF
Model(s)
CHROM, POS, RhythmFormer, 1D ResNet, CLIP ViT-L/14, Xception, DeepFakesON-Phys, LivePortrait, Thin-Plate Spline Motion Model (TPSMM)
Author countries
UNKNOWN