Hierarchical Deep Fusion Framework for Multi-dimensional Facial Forgery Detection - The 2024 Global Deepfake Image Detection Challenge

View on arXiv ← Back to list

Authors: Kohou Wang, Huan Hu, Xiang Liu, Zezhou Chen, Ping Chen, Zhaoxiang Liu, Shiguo Lian

Published: 2025-09-16 14:06:54+00:00

Comment: The 2024 Global Deepfake Image Detection Challenge Top20 Reward, 5 pages

AI Summary

This paper introduces the Hierarchical Deep Fusion Framework (HDFF), an ensemble-based deep learning architecture for high-performance facial forgery detection. HDFF integrates four diverse pre-trained sub-models (Swin-MLP, CoAtNet, EfficientNetV2, DaViT) which are fine-tuned through a multi-stage process on the MultiFFDI dataset. This approach achieved a score of 0.96852 and secured 20th position in a competitive challenge, demonstrating the efficacy of hierarchical fusion for complex image classification tasks.

Abstract

The proliferation of sophisticated deepfake technology poses significant challenges to digital security and authenticity. Detecting these forgeries, especially across a wide spectrum of manipulation techniques, requires robust and generalized models. This paper introduces the Hierarchical Deep Fusion Framework (HDFF), an ensemble-based deep learning architecture designed for high-performance facial forgery detection. Our framework integrates four diverse pre-trained sub-models, Swin-MLP, CoAtNet, EfficientNetV2, and DaViT, which are meticulously fine-tuned through a multi-stage process on the MultiFFDI dataset. By concatenating the feature representations from these specialized models and training a final classifier layer, HDFF effectively leverages their collective strengths. This approach achieved a final score of 0.96852 on the competition's private leaderboard, securing the 20th position out of 184 teams, demonstrating the efficacy of hierarchical fusion for complex image classification tasks.

Key findings
The HDFF achieved a final score of 0.96852 on the competition's private leaderboard, securing the 20th position out of 184 teams. Key findings included the significant performance improvement gained from using a CosineAnnealingLR scheduler, and the critical necessity of full-network fine-tuning for sub-models to achieve optimal results in specialized tasks like facial forgery detection.
Approach
The Hierarchical Deep Fusion Framework (HDFF) uses an ensemble of four diverse pre-trained sub-models (Swin-MLP, CoAtNet, EfficientNetV2, DaViT) for feature extraction. These models are fine-tuned using a multi-stage strategy. The final feature representations from these sub-models are then concatenated and fed into a new fully connected layer, which acts as the ultimate classifier.
Datasets
MultiFFDI dataset
Model(s)
Swin-MLP, CoAtNet, EfficientNetV2, DaViT
Author countries
China
← Previous
Next →