Generalizable and Adaptive Continual Learning Framework for AI-generated Image Detection

Authors: Hanyi Wang, Jun Lan, Yaoyu Kang, Huijia Zhu, Weiqiang Wang, Zhuosheng Zhang, Shilin Wang

Published: 2026-01-09 07:01:22+00:00

Comment: Accepted by TMM 2025

AI Summary

This paper introduces a novel three-stage domain continual learning framework for AI-generated image detection, designed to continuously adapt to evolving generative models and enhance generalization. It combines parameter-efficient fine-tuning for a transferable offline detector, a data augmentation chain with Kronecker-Factored Approximate Curvature (K-FAC) for new knowledge acquisition and catastrophic forgetting mitigation. The framework also leverages linear interpolation based on Linear Mode Connectivity to balance plasticity and stability, achieving superior performance on a benchmark of 27 generative models.

Abstract

The malicious misuse and widespread dissemination of AI-generated images pose a significant threat to the authenticity of online information. Current detection methods often struggle to generalize to unseen generative models, and the rapid evolution of generative techniques continuously exacerbates this challenge. Without adaptability, detection models risk becoming ineffective in real-world applications. To address this critical issue, we propose a novel three-stage domain continual learning framework designed for continuous adaptation to evolving generative models. In the first stage, we employ a strategic parameter-efficient fine-tuning approach to develop a transferable offline detection model with strong generalization capabilities. Building upon this foundation, the second stage integrates unseen data streams into a continual learning process. To efficiently learn from limited samples of novel generated models and mitigate overfitting, we design a data augmentation chain with progressively increasing complexity. Furthermore, we leverage the Kronecker-Factored Approximate Curvature (K-FAC) method to approximate the Hessian and alleviate catastrophic forgetting. Finally, the third stage utilizes a linear interpolation strategy based on Linear Mode Connectivity, effectively capturing commonalities across diverse generative models and further enhancing overall performance. We establish a comprehensive benchmark of 27 generative models, including GANs, deepfakes, and diffusion models, chronologically structured up to August 2024 to simulate real-world scenarios. Extensive experiments demonstrate that our initial offline detectors surpass the leading baseline by +5.51% in terms of mean average precision. Our continual learning strategy achieves an average accuracy of 92.20%, outperforming state-of-the-art methods.

Key findings

The initial offline detectors demonstrated strong generalization, outperforming the leading baseline by +5.51% in mean average precision. The proposed continual learning strategy achieved an average accuracy of 92.20%, surpassing state-of-the-art methods in adapting to evolving generative models. Furthermore, the method exhibited better overall robustness against common post-processing operations like Gaussian blur and JPEG compression.

Approach

The proposed approach involves a three-stage domain continual learning framework. The first stage develops a generalizable offline detector using parameter-efficient fine-tuning (LoRA) on the MLP layers of a CLIP Vision Transformer. The second stage integrates unseen data streams, employing a progressively complex data augmentation chain for plasticity and Kronecker-Factored Approximate Curvature (K-FAC) for stability to prevent catastrophic forgetting. Finally, the third stage utilizes a linear interpolation strategy based on Linear Mode Connectivity to capture commonalities across diverse generative models, enhancing overall performance by balancing plasticity and stability.

Datasets

A comprehensive benchmark of 27 generative models, chronologically structured from October 2017 to August 2024, including: GANs (ProGAN, CycleGAN, StarGAN, BigGAN, StyleGAN, GauGAN, StyleGAN2), Deepfakes (Deepfake), and Diffusion Models (DDPM, ADM, iDDPM, DALL·E, GLIDE, LDM, PNDM, Wukong, SD v1.4, Midjourney, SD v1.5, VQDM, SD v2.1, SDXL v1.0, SD-Turbo, SDXL-Turbo, SD v3.0, PixArt-Σ, FLUX.1).

Model(s)

The core architecture used is CLIP Vision Transformer (CLIP-ViT), specifically ViT-B/32 and ViT-L/14, as the backbone. For fine-tuning, Low-Rank Adaptation (LoRA) is employed. The continual learning process incorporates a Data Augmentation Chain, Kronecker-Factored Approximate Curvature (K-FAC) for Hessian approximation, and a Linear Mode Connectivity-based linear interpolation strategy.

Author countries

China

← Previous