Continuously Evolving Deepfake Detection: An Architecture and Public-Benchmark Evaluation of a Dynamic Detection System

Authors: Ken Jon Miyachi, Dylan Uys

Published: 2026-07-14 19:53:09+00:00

Comment: 16 pages, 1 figure

AI Summary

This paper introduces BitMind Forensics (BMF), a deepfake detection system designed to combat the non-stationary nature of deepfake generation by using a continuously evolving training distribution. Trained through an open adversarial competition (Bittensor SN34), BMF achieves high performance on various benchmarks, demonstrating robustness against real-world manipulations and AI-generated media. The system shows significant improvement over static detectors, particularly on recent and in-the-wild deepfake content.

Abstract

Deepfake detectors that achieve near-perfect scores on academic benchmarks collapse on real-world content: recent in-the-wild evaluations report AUC drops of 45-50% for state-of-the-art open-source models. We argue this gap is structural: static detectors are trained once against a moving generative frontier. We present BitMind Forensics (BMF), trained through Bittensor SN34, an open adversarial competition that continually refreshes the training distribution. We evaluate one dated export comprising image, general-video, and human-video checkpoints across nineteen public datasets: the canonical face-swap suites (FaceForensics++, Celeb-DF v1/v2/++, DFDC, DFD, UADFV, DF40) and recent in-the-wild and AI-generated-media benchmarks (Sumsub, Deepfake-Eval-2024, WildRF, Community Forensics, AIGCDetectBench, GenImage, AI-GenBench, AIGIBench, RAID, GenVidBench, GenVideo-100K). BMF reaches 0.936 AUC on Sumsub's original images and 0.872 pooled AUC over its full four-condition manipulation battery (1.4M images), staying robust under perturbation (0.855 JPEG, 0.799 downscaled), while GPEN enhancement improves detection (0.996). On Deepfake-Eval-2024, it matches the best commercial detector on images (0.915 vs 0.90) and exceeds it on video (0.822 vs 0.79), far above the best open-source detectors (0.56 and 0.63). It reaches 0.991 AUC on a 21-generator AI-image panel and 0.918 on GenVidBench, and exceeds the FF++-trained frontier on DFDC (0.947 vs 0.843) and Celeb-DF v2 (0.9985 vs 0.956), both contamination-audited, with statistical parity on Celeb-DF++. In a temporal study, successive dated exports improve on held-out media from generators absent from the static baseline's training (image 0.842 to 0.902; video 0.864 to 0.936). Our evaluation harness is public, and at publication the production API serves the exact evaluated snapshot for independent verification.

Key findings

BMF achieves 0.936 AUC on Sumsub original images and 0.872 pooled AUC over its full manipulation battery, showing robustness to JPEG compression (0.855 AUC) and downscaling (0.799 AUC). On Deepfake-Eval-2024, it matches the best commercial detector on images (0.915 AUC) and exceeds it on video (0.822 AUC). A temporal study demonstrates that successive exports of BMF significantly improve performance on media from generators absent in static baseline training (image AUC 0.842 to 0.902; video AUC 0.864 to 0.936).

Approach

BMF employs a dynamic architecture trained via Bittensor SN34, an adversarial competition that continuously updates its training distribution with new generative models. This process informs the development of an image ensemble and separate general and human-video checkpoints. These checkpoints leverage pretrained vision backbones, neural ensembles, and temporal adapters to detect deepfakes across diverse modalities and manipulation conditions.

Datasets

FaceForensics++, Celeb-DF v1/v2/++, DFDC, DFD, UADFV, DF40, Sumsub, Deepfake-Eval-2024, WildRF, Community Forensics, AIGCDetectBench, GenImage, AI-GenBench, AIGIBench, RAID, GenVidBench, GenVideo-100K, GAS-Station

Model(s)

ConvNeXt-L, EVA-L/14, CLIP ViT-L/14, DINOv3 ViT-L/16, V-JEPA 2, MTCNN (for face detection)

Author countries

UNKNOWN

← Previous