Bridging the SEA Gap: An Initial Benchmark for Neural Audio Codec-Synthesized Speech Deepfakes in South-East Asian Languages

Authors: Orchid Chetia Phukan, Girish, Mohd Mujtaba Akhtar, Arun Balaji Buduru

Published: 2026-06-14 18:50:14+00:00

Comment: Accepted to IJCAI-ECAI 2026

AI Summary

This paper introduces SEA-CF, the first large-scale benchmark for Neural Audio Codec-synthesized speech deepfake (Codecfake) detection in South-East Asian (SEA) languages. It demonstrates that existing state-of-the-art detectors fail to generalize to SEA speech and that large Audio Language Models (ALMs) are impractical due to their scale. To address this, the authors propose GARUDA, a novel, lightweight Small-ALM that achieves state-of-the-art performance in CF detection across SEA languages and existing benchmarks.

Abstract

Codecfakes (CFs) are a type of speech deepfakes generated through Audio Language Models (ALMs), with Neural Audio Codecs (NACs) forming the core mechanism for speech encoding and generation. CFs exhibit distributional characteristics that differ from vocoder-based deepfakes, causing detectors trained on vocoder data to generalize poorly to CFs detection. Although this has led to the development of CF detection benchmarks, existing resources are largely confined to English -- and to a limited extent Chinese -- leaving South-East Asian (SEA) languages unexplored. To bridge this gap, we introduce SEA-CF, the first large-scale benchmark for CF detection spanning multiple SEA languages, diverse speaker profiles, and a wide range of NAC architectures. SEA-CF is constructed by synthesizing publicly available real speech corpora. Our experiments show that state-of-the-art (SOTA) CF detectors trained on English-centric datasets fail to generalize to SEA speech due to language-specific phonetic structures, tonal variations, and rich prosodic diversity. We further conduct a comprehensive zero-shot and fine-tuned evaluation of recent SOTA ALMs on SEA-CF. Fine-tuning the ALMs improves performance, however, these are very large being impractical for real-world application due to their scale, particularly in low-resource and latency-constrained settings. To address this limitation, we propose a novel small-ALM, GARUDA tailored for CF detection, which delivers strong performance while remaining lightweight. Extensive evaluations demonstrate that the proposed Small-ALM outperforms strong end-to-end and ALM-based baselines, establishing a new, practical direction for robust CF detection in SEA languages and beyond.

Key findings

State-of-the-art Codecfake detectors trained on English-centric datasets exhibit poor generalization to South-East Asian languages due to linguistic differences, highlighting the necessity of in-domain training. The proposed GARUDA Small-ALM consistently outperforms larger state-of-the-art ALMs and traditional baselines across both SEA-CF and existing CodecFake benchmarks. GARUDA achieves superior performance with significantly fewer parameters and lower inference latency, establishing a practical and robust direction for CF detection in low-resource settings.

Approach

The authors introduce SEA-CF, a new benchmark for codec-fake detection in South-East Asian languages, generated by resynthesizing real speech using various Neural Audio Codecs (NACs). They propose GARUDA, a lightweight Small-ALM, which employs a dual-encoder design (Whisper and x-vector) to capture complementary speech representations, fuses them with Jensen–Shannon (JS) divergence for alignment, and feeds them into a Qwen2-0.5B language model decoder for classification.

Datasets

SEA-CF (constructed from Mozilla Common Voice, GigaSpeech2, Thai Dialect Corpus, VIVOS, Conversational Malay Speech Corpus, Malaysian YouTube dataset, Indic-SUPERB), CodecFake [Lu et al., 2024a]

Model(s)

GARUDA (proposed Small-ALM with Whisper and x-vector encoders, and Qwen2-0.5B LM decoder), Qwen-Audio-Chat, Qwen-Audio-Base, Qwen2-Audio-Chat, Qwen2-Audio-Base, SeaLLMs-Audio-7B, AASIST, Wh-LCNN, Wav2vec2-AASIST, MiO

Author countries

India

← Previous