Cross-Domain Generalization Limits of Vision Foundation Models in Facial Deepfake Detection

Authors: Ibrahim Delibasoglu

Published: 2026-05-24 09:33:46+00:00

AI Summary

This work systematically investigates the cross-domain generalization capabilities of Vision Foundation Models (VFMs) for detecting facial deepfakes, evaluating supervised (RoPE-ViT), self-supervised (DINOv3), and multi-teacher (NVIDIA C-RADIOv4-H) paradigms. By employing frozen backbones with downstream linear probing on the DF40 benchmark, the study reveals that while VFMs are effective for entire face synthesis, they struggle significantly with localized face editing techniques. The findings highlight the intrinsic trade-offs between pre-training paradigms and parameter scale, proving multi-teacher models are most resilient under extreme domain shifts.

Abstract

The rapid evolution of generative models has enabled the creation of hyper-realistic facial deepfakes, exposing a critical vulnerability in modern digital forensics: the inability of detectors to generalize to unseen manipulation techniques. Traditional networks suffer from representation collapse, overfitting to localized artifact fingerprints of specific training generators. This work investigates whether modern Vision Foundation Models can serve as generalizable, out-of-the-box feature extractors capable of tracking forensic anomalies across entirely unseen generative manifolds. We conduct a systematic cross-domain evaluation comparing three foundational learning paradigms: fully supervised macro-semantic features (RoPE-ViT), pure self-supervised geometric features (DINOv3), and multi-teacher agglomerative representations (NVIDIA C-RADIOv4-H). By deploying frozen backbones subjected to downstream linear probing, we map the performance limitations of these architectures on the challenging DF40 benchmark. Our empirical findings expose the intrinsic trade-offs between pre-training paradigms and parameter scale, proving that while foundation models retain high discriminative capabilities for entire face synthesis, localized face editing techniques expose fundamental boundaries in linear probe evaluation structures. Source code and model weights are available in http://github.com/mribrahim/deepfake

Key findings

Foundation models, especially DINOv3 and Supervised ViT, show strong discriminative capabilities for entire face synthesis deepfakes. However, they experience significant performance degradation when confronted with localized face editing techniques due to issues like low native patch resolution. The agglomerative multi-teacher representation (NVIDIA C-RADIOv4-H) proved most resilient under extreme domain shifts, retaining edge and semantic boundaries where other models failed.

Approach

The authors evaluate three distinct Vision Foundation Model (VFM) paradigms—fully supervised (RoPE-ViT), pure self-supervised (DINOv3), and multi-teacher agglomerative (NVIDIA C-RADIOv4-H)—by deploying their frozen backbones. A lightweight linear probe is then trained on the extracted features for binary deepfake classification. The approach systematically maps the performance limitations and generalization capabilities of these architectures on the DF40 benchmark.

Datasets

CelebA-HQ, FFHQ, LaPa, 100KFake, ThisPersonDoesNotExist, DF40 benchmark (including CollabDiff, StyleCLIP, MidJourney, WhichFaceIsReal)

Model(s)

RoPE-ViT (vit base patch16 rope mixed ape 224.naver in1k), DINOv3 (vit large patch16 dinov3.lvd1689m), NVIDIA C-RADIOv4-H, InceptionResNetV1, FreqNet, EfficientNet-B0, EfficientNet-B2, BNN

Author countries

Türkiye

← Previous