CUPID: Reconstructing UV Texture Maps for Interpretable Person-of-Interest Deepfake Detection

Authors: Giovanni Affatato, Sara Mandelli, Edoardo Daniele Cannas, Paolo Bestagini, Stefano Tubaro

Published: 2026-06-18 14:37:02+00:00

AI Summary

CUPID is a Person-of-Interest (POI) video deepfake detector that combines UV texture maps derived from 3D face reconstructions with a Masked Autoencoder (MAE) for representation learning. The method is trained solely on real videos and can detect deepfakes by matching embeddings from a query video against pristine reference videos of the POI. It also offers interpretability by highlighting manipulated facial regions, and demonstrates superior robustness to post-processing and faster inference compared to state-of-the-art approaches.

Abstract

Deepfakes targeting a high-profile individual, known as Person-of-Interest (POI), are a threat to modern democracies and societies. Current POI deepfake detection methods still struggle to combine robustness to post-processing, efficiency and interpretability, focal aspects of modern deepfake detectors. In this paper we propose CUPID, a POI video deepfake detector that combines UV texture maps, a facial appearance representation derived from 3D face reconstructions, with the representation learning capabilities of the Masked Autoencoder (MAE). Our method does not require any deepfake videos in its training phase. Moreover, it does not even require to include a specific POI in the training set: the combination of UV texture maps extracted from real video frames and the MAE context-guided reconstruction yields a latent space that captures rich and discriminative facial features also for identities unseen during training. In the testing phase, the embeddings extracted from a query video depicting the POI can be matched against pristine reference videos to assess the video authenticity. Furthermore, operating in the UV space naturally provides an additional layer of interpretability. Specifically, we can extract decoded residual maps that highlight which facial regions of a test video deviate most from the identity representation of the corresponding POI. Experiments on four deepfake datasets show that CUPID outperforms current state of the art on most datasets and achieves the best overall robustness against strong downscaling and compression, providing also substantially faster inference. Our experimental code will be released at https://github.com/polimi-ispl/CUPID.

Key findings

CUPID consistently outperforms current state-of-the-art deepfake detection methods on most datasets, demonstrating superior robustness against strong downscaling and compression while achieving substantially faster inference. The method also exhibits more stable threshold calibration across subjects and provides interpretable residual maps that highlight manipulated facial regions, thus offering insights into the detector's decisions.

Approach

The method extracts UV texture maps from 3D face reconstructions of input video frames. These UV maps are then fed into a Masked Autoencoder (MAE), which is self-supervisedly trained only on real videos using a combined reconstruction, contrastive, and perceptual loss to learn identity-aware latent representations. During inference, embeddings from a test video are compared against a set of pristine reference video embeddings of the POI to assess authenticity, and interpretability maps are generated by analyzing latent space deviations.

Datasets

VoxCeleb2 (training); DF-TIMIT, FakeAVCeleb, KoDF, DeepSpeak (testing)

Model(s)

Masked Autoencoder (MAE) built on Vision Transformer (ViT), 3D Morphable Models (3DMMs) for UV texture map extraction (using RetinaFace and 3DDFA-V3), and VGG-16 for perceptual loss.

Author countries

Italy

← Previous