Decoding while Adapting: Zero-Shot Online Speaker Adaptation via Audio-Textual Prompts for Elderly Speech Recognition
A novel approach for zero-shot, real-time speaker adaptation for elderly speech recognition.
The proposed approach enables zero-shot, real-time adaptation to unseen speakers, which is a novel contribution compared to prior work.
Before reading this…
Applications
- →elderly speech recognition
- →speech recognition in real-time
To understand this paper, make sure you know these concepts first:
- speech recognitionfind papers →
- speaker adaptationfind papers →
Abstract
More Like ThisThis paper proposes a novel cross-utterance audio-textual prompts based speaker adaptation approach for elderly speech recognition. It enables zero-shot, real-time adaptation to unseen speakers. Speech and text embeddings are extracted from the current and a few preceding utterances, before being fused in a cross-modal manner to produce compact speaker prompts that are more consistent than i/x-vectors and ECAPA-TDNN features. Experiments on the English DementiaBank Pitt and Cantonese JCCOCC MoCA elderly speech datasets suggest that the proposed online adaptation outperforms the speaker-independent (SI) model by statistically significant word error rate (WER) or character error rate (CER) reductions of 0.61% and 1.22% absolute (2.99% and 4.48% relative). Real-time factor (RTF) speed-up ratios of up to 9.83 times are obtained over offline batch-mode adaptation.