KanAdapter: A Kolmogorov-Arnold Network-based Plug-and-Play Module for Efficient Fine-tuning of Foundation Speech Models

View on arXiv ← Back to list

Authors: Phuong Tuan Dat, Phuong Khai Minh, Tran Huy Dat

Published: 2026-09-04 15:33:54+00:00

Comment: Accepted to IMPACT-SPEECH Workshop @ EMNLP 2026

AI Summary

The KanAdapter proposes a lightweight parameter-efficient fine-tuning framework for speech foundation models by replacing conventional MLP bottlenecks with Group-Rational Kolmogorov-Arnold Network (GR-KAN) modules. This approach significantly reduces trainable parameters while maintaining competitive performance across various speech tasks, including deepfake detection, speaker verification, and speech emotion recognition. KanAdapter also demonstrates superior resistance to catastrophic forgetting in continual learning scenarios.

Abstract

Fully fine-tuning self-supervised learning (SSL) speech models for downstream tasks is computationally prohibitive, and existing parameter-efficient fine-tuning approaches predominantly rely on MLP-based adapters whose fixed activation functions limit their representational expressiveness under tight parameter budgets. We propose \\textbf{KanAdapter}, a lightweight adapter framework that replaces conventional MLP bottlenecks with Group-Rational Kolmogorov-Arnold Network (GR-KAN) modules for more expressive and parameter-efficient adaptation. Following a parallel bottleneck design, KanAdapter inserts trainable GR-KAN branches alongside frozen Transformer encoder blocks and leverages weight transfer from pre-trained MLP layers for stable initialization. Across speaker verification, speech emotion recognition, and deepfake detection, KanAdapter achieves up to 97.5\\% reduction in trainable parameters relative to full fine-tuning while remaining highly competitive, and consistently outperforms AdaptFormer under comparable parameter budgets. In continual learning, it yields up to 83.6\\% error reduction over full fine-tuning and MLP-based adapters, which we attribute to the localized nature of GR-KAN's rational activations that mitigates catastrophic forgetting. To our knowledge, this is the first work to explore KAN-based modules for parameter-efficient fine-tuning of speech foundation models.

Key findings
KanAdapter achieves up to a 97.5% reduction in trainable parameters compared to full fine-tuning while maintaining highly competitive performance across speaker verification, speech emotion recognition, and deepfake detection tasks. It consistently outperforms AdaptFormer under comparable parameter budgets. In continual learning, KanAdapter shows up to an 83.6% error reduction over full fine-tuning and MLP-based adapters, attributed to the localized nature of GR-KAN's rational activations, which mitigates catastrophic forgetting.
Approach
KanAdapter integrates Group-Rational Kolmogorov-Arnold Network (GR-KAN) modules as plug-in adapters within frozen Self-Supervised Learning (SSL) backbones. It employs a parallel bottleneck design where trainable GR-KAN branches are inserted alongside frozen Transformer encoder blocks. Weight transfer from pre-trained MLP layers is used for stable initialization.
Datasets
VoxCeleb2, VoxCeleb1, MSP-Podcast, ASVspoof2019 LA, ASVspoof2021 LA, ASVspoof2021 DF, ASVspoof5, In-The-Wild
Model(s)
WavLM, XLS-R, WavLM-ECAPA, WavLM with attentive statistical pooling, XLSR-Conformer, UniSpeech-SAT, mHuBERT-147
Author countries
Singapore
← Previous
Next →