Transferable Class-Modelling for Decentralized Source Attribution of GAN-Generated Images

View on arXiv ← Back to list

Authors: Brandon B. G. Khoo, Chern Hong Lim, Raphael C. -W. Phan

Published: 2022-03-18 07:43:03+00:00

AI Summary

This paper proposes a semi-decentralized framework for detecting and attributing GAN-generated images by redefining the problem as a series of binary classification tasks. It leverages transfer learning to efficiently adapt forgery detection networks for multiple attribution problems, improving scalability and model interpretation.

Abstract

GAN-generated deepfakes as a genre of digital images are gaining ground as both catalysts of artistic expression and malicious forms of deception, therefore demanding systems to enforce and accredit their ethical use. Existing techniques for the source attribution of synthetic images identify subtle intrinsic fingerprints using multiclass classification neural nets limited in functionality and scalability. Hence, we redefine the deepfake detection and source attribution problems as a series of related binary classification tasks. We leverage transfer learning to rapidly adapt forgery detection networks for multiple independent attribution problems, by proposing a semi-decentralized modular design to solve them simultaneously and efficiently. Class activation mapping is also demonstrated as an effective means of feature localization for model interpretation. Our models are determined via experimentation to be competitive with current benchmarks, and capable of decent performance on human portraits in ideal conditions. Decentralized fingerprint-based attribution is found to retain validity in the presence of novel sources, but is more susceptible to type II errors that intensify with image perturbations and attributive uncertainty. We describe both our conceptual framework and model prototypes for further enhancement when investigating the technical limits of reactive deepfake attribution.

Key findings

The proposed model shows competitive performance with existing benchmarks in deepfake detection, and demonstrates efficiency gains in source attribution. However, the accuracy of source attribution decreases significantly with image perturbations, highlighting the vulnerability of fingerprint-based methods.

Approach

The authors propose a modular framework where a primary module detects deepfakes, and secondary modules, trained via transfer learning from the primary module, attribute images to specific GANs. This approach uses binary classification for source attribution, simplifying the problem and improving scalability.

Datasets

FacesHQ+ (extended with StyleGAN2 and StarGANv2 images), GAN Fingerprints (GANFP) dataset (truncated to 20% of original size)

Model(s)

Custom convolutional neural networks (CNNs) with a primary module for deepfake detection and multiple secondary modules for source attribution. Baseline models gandct-conv and ganfp-postpool were also used for comparison.

Author countries

Malaysia

← Previous