Towards Zero-Shot Attribution of Synthetic Speech via Audio-Text Contrastive Retrieval
This paper proposes a new approach to audio deepfake attribution that can identify the system that generated an audio clip without retraining or updating the classifier.
This paper introduces a new approach to audio deepfake attribution that can identify the system that generated an audio clip without retraining or updating the classifier.
Before reading this…
Applications
- →Audio forensics
- →Deepfake detection
To understand this paper, make sure you know these concepts first:
- Knowledge of deep learning and audio processingfind papers →
- Understanding of text-to-speech systemsfind papers →
Abstract
More Like ThisAudio deepfake forensics is moving beyond a simple real-or-fake verdict toward attribution: which system generated the audio clip? Most source-attribution methods cast this as closed-set classification, so they cannot name a generator that was absent from training, a gap that widens with every newly released text-to-speech (TTS) system. We instead frame attribution as cross-modal retrieval: each generator is described in natural language, and a clip is attributed by retrieving the description closest to it in a shared audio-text embedding space. Adding a new system then takes nothing more than writing its description, with no retraining and no new classifier head. Our model couples a frozen Wav2Vec2-BERT audio encoder with a frozen E5 text encoder and aligns them through small trainable projection heads, using a contrastive objective that combines a cross-modal supervised-contrastive loss with intra-modal terms. We evaluate on MLAAD v9 (140 TTS models, 51 languages) under 10-fold leave-models-out cross-validation. For generators it has never encountered before, the model reaches a model-level mean reciprocal rank (MRR) of 58.4%. Even when the correct model is not identified, the audio clip is often matched to systems that share the true generator's vocoder, acoustic model, or architecture. Because the same embedding space also answers natural-language attribute queries, one set of descriptions covers both open-set attribution and attribute-level forensic profiling.