FlowTTS-GRPO: Online Reinforcement Learning with Multi-Objective Reward Optimization for Flow-Matching Based Text-to-Speech
This paper presents FlowTTS-GRPO, an online RL framework for Flow-Matching based Text-to-Speech that enables direct fine-tuning of open-source models.
This paper is novel in its application of online RL to Flow-Matching based TTS, enabling direct fine-tuning of open-source models.
Keywords
Before reading this…
Applications
- →Text-to-Speech systems
- →Speech synthesis
To understand this paper, make sure you know these concepts first:
- Reinforcement Learningfind papers →
- Text-to-Speechfind papers →
Abstract
More Like ThisExisting Reinforcement Learning (RL) research for Text-to-Speech (TTS) focuses on large language models (LLMs), leaving Flow-Matching (FM) under-explored. We present FlowTTS-GRPO, an online RL framework for FM-based TTS. By converting ordinary differential equation (ODE) trajectories into stochastic differential equation (SDE) paths, our method enables direct fine-tuning of open-source FM models without auxiliary models. We show that a weighted reward combination converges faster than a probabilistic scheme, and identify three practical optimizations: omitting classifier-free guidance (CFG) during training accelerates convergence; synthesizing hard cases improves robustness; and applying RL to the FM component enhances audio-detail metrics. Experiments on CosyVoice 3.0 and F5-TTS demonstrate objective and subjective preference gains in speaker similarity and perceptual quality, with F5-TTS also improving intelligibility.