Transferable Class-Modelling for Decentralized Source Attribution of GAN-Generated Images

Authors: Brandon B. G. Khoo, Chern Hong Lim, Raphael C. -W. Phan

Published: 2022-03-18 07:43:03+00:00

Comment: 21 pages, 8 figures. Code: https://github.com/quarxilon/Generator_Attribution

AI Summary

This paper redefines GAN-generated deepfake detection and source attribution as a series of related binary classification tasks to enhance scalability and functionality. It proposes a semi-decentralized modular design that leverages transfer learning to rapidly adapt forgery detection networks for multiple independent attribution problems. The approach demonstrates competitive performance and the validity of decentralized fingerprint-based attribution even for novel sources, though it is susceptible to image perturbations.

Abstract

GAN-generated deepfakes as a genre of digital images are gaining ground as both catalysts of artistic expression and malicious forms of deception, therefore demanding systems to enforce and accredit their ethical use. Existing techniques for the source attribution of synthetic images identify subtle intrinsic fingerprints using multiclass classification neural nets limited in functionality and scalability. Hence, we redefine the deepfake detection and source attribution problems as a series of related binary classification tasks. We leverage transfer learning to rapidly adapt forgery detection networks for multiple independent attribution problems, by proposing a semi-decentralized modular design to solve them simultaneously and efficiently. Class activation mapping is also demonstrated as an effective means of feature localization for model interpretation. Our models are determined via experimentation to be competitive with current benchmarks, and capable of decent performance on human portraits in ideal conditions. Decentralized fingerprint-based attribution is found to retain validity in the presence of novel sources, but is more susceptible to type II errors that intensify with image perturbations and attributive uncertainty. We describe both our conceptual framework and model prototypes for further enhancement when investigating the technical limits of reactive deepfake attribution.

Key findings

The proposed modular binary classification framework achieved competitive performance for deepfake detection and improved training efficiency for source attribution compared to baselines. However, source attribution proved significantly more challenging than deepfake detection, showing reduced sensitivity and increased susceptibility to image perturbations and similar GM fingerprints, especially for high-resolution images where features failed to generalize well.

Approach

The authors redefine deepfake detection and source attribution as multiple binary classification tasks. They propose a semi-decentralized modular ConvNet design that utilizes transfer learning: a primary module, pre-trained for source-agnostic deepfake detection, serves as a feature extractor for specialized secondary modules that perform one-versus-all binary attribution for specific generative models. Class activation mapping is also incorporated for model interpretation and feature localization.

Datasets

FacesHQ+ (derived from CelebA-HQ, FFHQ, ThisPersonDoesNotExist.com (TPDNE), generated.photos project), StarGANv2 (out-of-distribution samples), GAN Fingerprints (GANFP) dataset (including CelebA, SNGAN, ProGAN, MMDGAN, CramerGAN generated images).

Model(s)

The proposed model is a modular convolutional neural network (ConvNet) featuring double strided convolutions, batch normalization, global average pooling, and Class Activation Mapping (CAM) or Grad-CAM. Baseline models include `gandct-conv` (from Frank et al., 2020) and a scaled-down `ganfp-postpool` (from Yu et al., 2019).

Author countries

Malaysia, Australia

← Previous