Inherited Adapter Experts for Replay-Free Domain-Incremental Learning

Authors: Gyutae Oh, Hyunmin Kong, Jitae Shin

Published: 2026-11-18 09:53:15+00:00

Comment: 21 pages (8 pages main text, plus references and supplementary material), 8 figures, 23 tables

AI Summary

This paper introduces Lamarck, a novel replay-free domain-incremental learning method that enhances expert accuracy by utilizing adapters in deep Vision Transformer blocks and initializing new experts from their predecessors. Unlike prior work focusing on router training, Lamarck prioritizes strengthening the experts and employs a training-free Mahalanobis router. This approach significantly improves performance on deepfake detection and other benchmarks.

Abstract

Replay-free domain-incremental learning trains a model on the same classes in successive domains without keeping earlier images. Per-domain learners freeze a pre-trained ViT and train an additional expert for each new domain: the small set of parameters and the classification head learned on that domain. Test images carry no domain label, so a router selects one expert for each image. Recent work trains this router. However, a training-free Mahalanobis router costs the analyzed learners under 2.7 points of final accuracy relative to always selecting each image's own-domain expert. The experts, not this router, limit accuracy. Our learner, Lamarck, therefore uses this router and strengthens the experts. Its adapters, small trainable layers that correct every image token after each deep ViT block, add 4.4 points of final accuracy on the deepfake benchmark CDDB-Hard. In addition, each expert is initialized from its predecessor at no extra cost: the domains share their classes, so an image misrouted to a later expert loses less. Against twelve frozen-backbone baselines under one protocol, Lamarck is first on CDDB-Hard by 5.2 points, second on DomainNet and OfficeHome, and highest in mean accuracy.


Key findings
Lamarck achieves state-of-the-art performance, leading on the CDDB-Hard deepfake benchmark by 5.2 points and having the highest mean accuracy across three benchmarks. The adapters contribute a significant accuracy gain (e.g., 4.4 points on CDDB-Hard), and inheritance further lowers the accuracy loss when images are misrouted to later experts. The training-free Mahalanobis router incurs minimal accuracy loss compared to the gains achieved by stronger experts.
Approach
Lamarck strengthens per-domain experts by adding residual bottleneck adapters after each deep ViT block to correct image tokens and initializing each new expert from its predecessor. It employs a training-free Mahalanobis router to select the appropriate expert for each test image based on feature similarity, focusing on improving expert accuracy rather than router performance.
Datasets
CDDB-Hard, DomainNet, OfficeHome
Model(s)
ViT-B/16, DINOv2 ViT-B/14 (for ablation)
Author countries
Republic of Korea