Anchoring the Unknown: Open-Set Model Attribution via Proxy-Anchor Learning

Authors: Cristian-Teodor Neamtu, Serban Mihalache, Stefan Smeu, Dan Oneata, Horia Cucu, Dragos Burileanu

Published: 2026-06-09 12:10:29+00:00

Comment: Accepted to the 34th European Signal Processing Conference (EUSIPCO 2026)

AI Summary

This paper proposes a metric learning framework for open-set text-to-speech (TTS) source attribution, addressing the challenge of identifying the generative system and detecting unknown ones. It leverages Wav2Vec2-BERT embeddings optimized with a Proxy-Anchor loss function and introduces an architecture merging strategy to reduce inter-class confusion. The approach demonstrates superior performance in both closed-set attribution and out-of-distribution detection, outperforming existing state-of-the-art methods.

Abstract

The proliferation of text-to-speech (TTS) systems capable of generating realistic synthetic speech poses growing challenges for audio forensics. While binary deepfake detection has received considerable attention, source tracing (i.e., identifying which TTS system produced a given audio sample) remains underexplored, particularly in open-set scenarios where unknown systems may be encountered. We propose a metric learning framework based on the Proxy-Anchor loss function that operates on Wav2Vec2-BERT embeddings to learn a discriminative embedding space for TTS source attribution and out-of-distribution (OOD) detection of unseen systems. We evaluate it on the MLAAD v9 dataset spanning 140 TTS systems across 51 languages, and introduce an architecture merging strategy that groups TTS system versions into unified classes, reducing inter-class confusion. Our system achieves 99.76% accuracy on 110 in-distribution classes and a False Positive Rate (FPR@95) as low as 2.04% for OOD detection. Also, for a fair comparison against the current state of the art, we further evaluate it on the MLAAD v5 official dataset splits, improving the OOD accuracy by almost doubling it. These results demonstrate that Proxy-Anchor metric learning, combined with architecture-aware class design and post-hoc OOD scoring, provides an effective framework for forensic TTS source tracing in both closed-set and open-set settings.

Key findings

The system achieved 99.76% accuracy on 110 in-distribution classes and a False Positive Rate (FPR@95) of 2.04% for OOD detection on MLAAD v9. When evaluated on MLAAD v5, it significantly improved OOD accuracy by nearly doubling it and reduced FPR@95 by 60% compared to the state-of-the-art. The study demonstrates that Proxy-Anchor metric learning, combined with architecture-aware class design and post-hoc OOD scoring, is highly effective for forensic TTS source tracing in both closed-set and open-set settings.

Approach

The problem is solved using a metric learning framework that processes Wav2Vec2-BERT embeddings through a linear projection head, optimized with the Proxy-Anchor loss function. This creates a discriminative embedding space where each TTS system class is represented by a learnable proxy. Inference involves a two-stage process: first, OOD detection using scoring functions (Softmax energy, Shannon entropy, or Maximum proxy distance), followed by in-distribution attribution by identifying the nearest class proxy.

Datasets

MLAAD v9, MLAAD v5

Model(s)

Wav2Vec2-BERT (feature extractor), Linear Projection Head, Proxy-Anchor Loss (for metric learning)

Author countries

Romania

← Previous