MOTIF: Person-of-Interest Deepfake Detection Beyond 3DMM Coefficients

View on arXiv ← Back to list

Authors: Giovanni Affatato, Sara Mandelli, Paolo Bestagini, Stefano Tubaro

Published: 2026-10-07 10:51:29+00:00

Comment: 6 pages. Accepted at the 2026 IEEE International Workshop on Information Forensics and Security (WIFS)

AI Summary

This paper introduces MOTIF, a visual-only Person-of-Interest (POI) deepfake detector that dissects the utility of 3D Morphable Model (3DMM) coefficients and the dense surface reconstruction for deepfake detection. It demonstrates that while 3DMM coefficients are largely redundant, the dense surface, particularly of the mouth, carries crucial identity information not captured by coefficients, significantly improving detection, especially where coefficient-based methods are weak. MOTIF, trained only on real videos without manipulated data or POI-specific information, outperforms existing state-of-the-art POI detectors across various datasets and quality levels.

Abstract

Video deepfakes targeting a specific individual, the Person-of-Interest (POI), are the most harmful ones, and, since a public figure is abundantly recorded, a detector can be built from genuine footage of that individual. Such detectors commonly describe a subject through a 3D Morphable Model (3DMM) and adopt its coefficients as a whole, so which part of that description carries the signal has never been measured. We dissect it, holding the encoder, the training corpus and the enrollment protocol fixed and varying only what the encoder observes. The groups of coefficients prove largely redundant, since the shape block alone recovers almost all the accuracy of the full vector, and their temporal evolution contributes a real but bounded amount. We further show that the dense surface the same fit returns, which these detectors discard, carries identity information that the coefficients do not, and that it helps precisely where they are weakest. We assemble the best configuration into MOTIF, a visual-only detector trained on real videos only, with no manipulated video and no POI-specific data. It improves on both state-of-the-art POI detectors in every dataset and manipulation of our benchmark and at two quality levels. Our experimental code will be released at https://github.com/polimi-ispl/MOTIF.

Key findings
The shape block of 3DMM coefficients alone recovers almost all the accuracy of the full vector, indicating redundancy in the coefficient groups, and their temporal evolution offers a real but limited contribution. The dense surface reconstruction, specifically the mouth region, carries unique identity information not captured by 3DMM coefficients and significantly enhances detection accuracy, particularly for lip-sync manipulations. MOTIF, integrating these findings, achieves superior performance compared to state-of-the-art POI deepfake detectors, especially under low-quality video conditions.
Approach
The approach analyzes 3DMM coefficients (shape, expression, pose, translation) and the dense 3D surface reconstruction of the face, specifically focusing on the mouth. It uses two separate temporal transformers: one for global face descriptors derived from 3DMM coefficients and another for mouth descriptors from the dense surface. These branches are trained using an identity-contrastive objective on real videos, and their scores are fused after per-POI calibration.
Datasets
VoxCeleb2 (training), DF-TIMIT, FakeAVCeleb, DeepSpeak, KoDF (evaluation).
Model(s)
3DDFA-V3 (for 3DMM coefficient extraction), Temporal Transformer (for encoding), Multi-Similarity Miner (for contrastive learning).
Author countries
Italy
← Previous
Next →