One-Step Token-to-Waveform Generation with MeanFlow in Latent Space
A novel Token2Wav architecture is proposed to improve the efficiency of LLM-based TTS and multimodal systems.
The novelty lies in the MeanFlow architecture, which enables true one-step generation and improves efficiency without sacrificing quality.
Before reading this…
Applications
- →LLM-based TTS
- →Multimodal systems
To understand this paper, make sure you know these concepts first:
- Neural audio codecsfind papers →
- LLM-based TTSfind papers →
Abstract
More Like ThisNeural audio codecs are central to modern LLM-based Text-to-Speech (TTS) and multimodal systems. As low-bitrate semantic codecs gain prominence, the Token-to-Waveform (Token2Wav) decoder becomes a bottleneck determining both perceptual quality and system efficiency. Conventional multi-step flow-matching decoders offer superior quality but suffer from high inference latency due to iterative sampling, creating a severe quality-speed trade-off. In this paper, we propose a novel Token2Wav architecture that overcomes this limitation by applying MeanFlow in a highly compressed latent space. By modeling the average velocity rather than the instantaneous velocity field, MeanFlow enables true one-step generation. Operating in the latent domain mitigates the memory and stability issues of waveform-level flows, yielding up to a 17$\times$ improvement in Real-Time Factor (RTF) compared to multi-step baselines with negligible quality degradation. Furthermore, we introduce refinement strategies that mitigate latent mismatch, including decoder-only fine-tuning with the MeanFlow generator frozen and end-to-end joint fine-tuning, improving fidelity without increasing inference-time cost. Code and demo are publicly available.