DFD-Lab: A Modular Audio-Visual Deepfake Detection Pipeline

Authors: Jan Rybarczyk, Mateusz Roszkowski, Jacek Komorowski

Published: 2026-09-20 19:29:40+00:00

AI Summary

DFD-Lab is a modular audio-visual deepfake detection pipeline designed for comparing detectors by coordinating dataset adaptation, temporal input representation, model interfaces, and experimental conditions. It integrates various deepfake detectors and supports flexible training and evaluation workflows, including external testing, degradation-based training augmentation, and evaluation-time corruption. The pipeline's empirical case studies highlight the challenges of cross-dataset detection and the importance of using both ranking and classification metrics.

Abstract

Comparing audio-visual deepfake detectors requires coordinating dataset adaptation, temporal input representation, model interfaces and experimental conditions. We present DFD-Lab, a modular pipeline that separates these responsibilities while supporting shared training and evaluation workflows. We integrate three implementations: Xception-based maximum-logit fusion, ResNet with temporal LSTM fusion, and our AVFF reimplementation. Experiments cover external testing, degradation-based training augmentation and evaluation-time corruption. On a filtered subset of Deepfake-Eval-2024, models trained on FakeAVCeleb attain baseline AUROC values of 0.504, 0.538 and 0.458. JPEG50 training augmentation raises these to 0.691, 0.605 and 0.570, respectively, while all three accuracies decrease. These results illustrate why training interventions, evaluation corruptions and metric-dependent outcomes should remain distinct within a common pipeline. The contribution is the integration of audio-visual processing, interchangeable detectors and configurable experimental workflows, supported by empirical case studies. The findings highlight the challenge of cross-dataset detection and the complementary information provided by ranking and classification metrics.

Key findings

JPEG50 training augmentation consistently improved external AUROC across all implemented detectors but simultaneously decreased their accuracy, highlighting the distinction between ranking and classification metrics. Evaluation-time corruption revealed varied sensitivities to degradations across models, with audio noise (SNR10) consistently reducing AUROC for all detectors. These results emphasize the importance of defining the role and placement of interventions (e.g., training vs. evaluation) and considering multiple evaluation metrics.

Approach

DFD-Lab provides a modular pipeline separating data preparation, detector computation, and experiment orchestration. It uses dataset-specific adapters, a shared clip representation, and a common detector interface. Experiments involve three audio-visual detector implementations (Xception Max-Fusion, ResNet+LSTM, and AVFF-R) evaluated across external testing, training augmentation with degradations, and evaluation-time corruption.

Datasets

FakeAVCeleb, Deepfake-Eval-2024 (filtered subset), LRS2 (for pretraining AVFF-R)

Model(s)

Xception-based maximum-logit fusion, ResNet with temporal LSTM fusion, AVFF reimplementation (AVFF-R)

Author countries

Poland