Leveraging Failed Samples: A Few-Shot and Training-Free Framework for Generalized Deepfake Detection

Authors: Shibo Yao, Renshuai Tao, Xiaolong Zheng, Chao Liang, Chunjie Zhang

Published: 2025-08-13 04:06:09+00:00

AI Summary

This paper introduces Few-shot Training-free Network (FTNet), a novel framework for generalized deepfake detection that re-conceptualizes the problem as a few-shot task. Unlike traditional methods requiring extensive training on large datasets, FTNet operates without parameter updates, leveraging only a minimal number of real and fake samples from an evaluation set to classify new deepfake images. It compares test samples to a dynamic knowledge base built with these few-shot samples, achieving state-of-the-art performance across various generative models.

Abstract

Recent deepfake detection studies often treat unseen sample detection as a ``zero-shot task, training on images generated by known models but generalizing to unknown ones. A key real-world challenge arises when a model performs poorly on unknown samples, yet these samples remain available for analysis. This highlights that it should be approached as a ``few-shot task, where effectively utilizing a small number of samples can lead to significant improvement. Unlike typical few-shot tasks focused on semantic understanding, deepfake detection prioritizes image realism, which closely mirrors real-world distributions. In this work, we propose the Few-shot Training-free Network (FTNet) for real-world few-shot deepfake detection. Simple yet effective, FTNet differs from traditional methods that rely on large-scale known data for training. Instead, FTNet uses only one fake samplefrom an evaluation set, mimicking the scenario where new samples emerge in the real world and can be gathered for use, without any training or parameter updates. During evaluation, each test sample is compared to the known fake and real samples, and it is classified based on the category of the nearest sample. We conduct a comprehensive analysis of AI-generated images from 29 different generative models and achieve a new SoTA performance, with an average improvement of 8.7\\% compared to existing methods. This work introduces a fresh perspective on real-world deepfake detection: when the model struggles to generalize on a few-shot sample, leveraging the failed samples leads to better performance.

Key findings

FTNet achieves new state-of-the-art performance, with an average improvement of 8.7% (FTNet-T up to 12.1% with fine-tuning) compared to existing methods. It demonstrates superior data efficiency, outperforming baselines trained on millions of images with only 4-shot or 8-shot adaptation. The method effectively captures universal forgery signatures in CLIP's intermediate features, even for completely unseen generative models and varying semantic content.

Approach

FTNet utilizes a pre-trained CLIP model (specifically its image encoder) to extract features from intermediate layers. A 'cache model' consisting of a Feature Bank (L2-normalized CLIP features) and a Label Bank (one-hot encoded labels) is built using a few-shot support set. During inference, a test image's features are compared to the Feature Bank via cosine similarity, and classification logits are derived by integrating labels from the Label Bank based on these similarities.

Datasets

GenImage, UniversalFakeDetect, OpenSDI, Chameleon, GANGen

Model(s)

CLIP (specifically the visual backbone, ViT-L/14)

Author countries

China

← Previous