FYC: FPR-Disparity Weight Perturbation and Youden Calibration for Fair Deepfake Detection

Authors: Uzoamaka Ezeakunne, Taiwo Olabamipe, Deevesh Jain, Xiuwen Liu

Published: 2026-10-27 19:22:21+00:00

Comment: 9 pages, 4 figures

AI Summary

This paper introduces FYC (FPR-disparity Weight Perturbation and Youden Calibration), a two-stage post-hoc method to address demographic disparities in deepfake detection. FYC perturbs detector weights based on an FPR-disparity criterion and then calibrates decision thresholds for each demographic group, requiring no architectural changes or full retraining. The method significantly outperforms existing approaches in fairness and utility metrics on the AI-Face v2 benchmark.

Abstract

Deepfake detectors routinely achieve high aggregate accuracy while exhibiting substantial demographic disparities in their errors: false-positive rates differ by double digits across gender and skin tone. Most existing remedies require either full retraining with a fairness-aware objective or demographic-annotated training data, both impractical when only a pretrained checkpoint is available, while label-free post-hoc calibrators leave the detector's weights unchanged. We introduce FYC (FPR-disparity Weight Perturbation and Youden Calibration), a two-stage post-hoc method that perturbs a detector's weights under an FPR-disparity criterion, then calibrates a separate decision threshold for each demographic group - requiring no architectural changes and no full retraining. Evaluated against every detector benchmarked on AI-Face v2 across six intersectional gender x skin-tone groups, FYC outperforms the field on 18 of 21 reported fairness and utility metrics, including purpose-built fairness architectures, at a fraction of the training cost. We further identify why the choice of fairness criterion is critical: the original Demographic Parity objective used in prior weight-perturbation work is architecture-dependent and unpredictable, improving two of four evaluated architectures, leaving a third largely unchanged, and substantially degrading the fourth (PG-FDD), with no way to anticipate the outcome for a given checkpoint in advance. Conditioning the same objective on real images only - our FPR-disparity criterion - eliminates this unpredictability, improving fairness and accuracy jointly across all four architectures with no exceptions.


Key findings
FYC applied to an Xception backbone outperforms 12 other detectors on AI-Face v2, winning 18 of 21 fairness and utility metrics with significantly reduced training cost. The FPR-disparity criterion is crucial, as it reliably improves fairness and accuracy across all tested architectures, unlike the unpredictable Demographic Parity objective. Weight perturbation combined with Youden-J calibration is decisive, with calibration alone being insufficient for the observed gains.
Approach
FYC is a two-stage post-hoc method. Stage 1 involves FPR-disparity Weight Perturbation, where a pretrained detector's weights are fine-tuned by perturbing them towards worst-case disparity, conditioned on real images only to ensure reliability across architectures. Stage 2 applies per-group Youden-J Calibration, which finds optimal decision thresholds for each demographic group on a held-out calibration split, further enhancing fairness and utility.
Datasets
AI-Face v2
Model(s)
Xception, EfficientNet-B4, DAG-FDD, PG-FDD
Author countries
United States