Low-Latency Real-Time Audio Game Commentary System via LLM-Based Parallel Text Generation
A low-latency real-time audio game commentary system generates spoken commentary directly from live gameplay video.
This paper presents a novel parallel design for real-time audio commentary that improves latency and similarity to professional speaking compared to sequential baselines.
Before reading this…
Applications
- →game commentary
- →sports commentary
To understand this paper, make sure you know these concepts first:
- natural language processingfind papers →
- speech synthesisfind papers →
Abstract
More Like ThisWe present a low-latency real-time audio game commentary system that generates spoken commentary directly from live gameplay video. In this end-to-end setting, a key bottleneck is accumulated waiting time; conventional pipelines capture frames, generate text, and synthesize speech sequentially for each utterance, and do not request the next generation until speech playback has completed. This strict sequentiality causes long and unnatural silence between utterances. To address this latency bottleneck, our system runs text generation in parallel with speech playback and buffers multiple candidate utterances ahead of time, enabling immediate synthesis at playback boundaries. Experiments on fast-paced game videos show that our parallel design reduces the mean inter-utterance silence from 9.6 seconds to 0.3 seconds compared to sequential baselines. It also improves similarity to professional speaking--silence timing patterns by over 40 %, and a user study with 120 experienced game players confirms significantly improved perceived speaking rhythm. Our demo video is available at: https://youtu.be/pmrRUlvav8M.