Toward Calibrated, Fair, and accurate Deepfake Detection

Authors: Ryan Brown, Chris Russell

Published: 2026-06-03 05:44:29+00:00

AI Summary

This paper introduces Face-Fairness (FF), a plug-and-play framework designed to mitigate bias and improve the accuracy of deepfake detectors, which often show large performance gaps across demographic groups. Its primary contribution, Face-Feature Tuning (FFT), is a novel demographic label-free fairness method. FFT employs a lightweight calibrator that performs a logit remapping conditioned on frozen face embeddings, reducing FPR/TPR gaps and improving minimum group accuracy while maintaining or improving overall accuracy without requiring demographic labels, retraining, or significant runtime overhead.

Abstract

Deepfake detectors show large performance gaps across demographic groups. Existing fairness approaches require demographic labels, retraining, or sacrifice accuracy. We introduce Face-Fairness (FF), a plug-and-play framework for bias mitigation. Our primary contribution, Face-Feature Tuning (FFT), is the first demographic label-free fairness method demonstrated for deepfake detection: a lightweight calibrator that performs a logit remapping conditioned on frozen face embeddings. We complement FFT with two variants: FF-Max, which maximizes worst-group accuracy when demographics are available, and FF-Discover, which does the same with embedding-discovered groups. Across in-domain and cross-dataset test settings, FF consistently reduces FPR/TPR gaps and improves minimum group accuracy while maintaining (often improving) overall accuracy. The approach is detector-agnostic, adds negligible runtime overhead, and requires no access to identity attributes.

Key findings

FFT consistently improves overall accuracy and fairness metrics (reduces FPR/TPR gaps, increases worst-group accuracy) across in-domain and cross-dataset test settings, outperforming other post-processing baselines. It is the only post-processing method that simultaneously improves accuracy and reduces fairness gaps on in-domain data, and effectively recovers above-chance accuracy under cross-generator distribution shifts. Furthermore, stacking FFT with existing in-processing fairness methods further enhances both fairness and utility, suggesting it corrects residual biases.

Approach

The Face-Fairness (FF) framework includes three variants: Face-Feature Tuning (FFT), FF-Max, and FF-Discover. FFT, the main contribution, is a post-processing calibration method that trains a shallow multi-layer perceptron (MLP) on top of frozen face embeddings (e.g., ArcFace) and the base detector's logit to learn a better decision boundary. FF-Max optimizes group-specific decision thresholds when demographic labels are available, while FF-Discover achieves this by automatically discovering latent groups through K-means clustering on face embeddings when labels are unavailable.

Datasets

OpenForensics, FaceForensics++ (FF++) with manipulation families DeepFakes (DF), FaceSwap (FS), FaceShifter (FST), Face2Face (F2F), and NeuralTextures (NT).

Model(s)

Xception, MobileNetV3-Small (as base deepfake detectors), ArcFace (for face embeddings), InsightFace's FaceAnalysis pipeline (for face detection), FairFace ResNet-34 (for demographic attribute classification).

Author countries

United Kingdom

← Previous