One Model, Many Latencies: Universal Speech Enhancement for Diverse Real-Time Applications
A universal speech enhancement model that provides explicit control over algorithmic and computational latency for various real-time speech applications.
The proposed model provides explicit control over both algorithmic and computational latency, which is novel compared to prior work.
Keywords
Before reading this…
Applications
- →real-time speech applications
To understand this paper, make sure you know these concepts first:
- speech enhancementfind papers →
- deep learningfind papers →
- convolutional neural networksfind papers →
Abstract
More Like ThisDifferent real-time speech applications impose distinct latency budgets, often requiring separately trained enhancement models for each scenario. In this paper, we propose a one-for-all, real-time universal speech enhancement model that provides explicit control over both algorithmic and computational latency. Algorithmic latency is flexibly adjusted via configurable look-ahead frames. To avoid learning inefficiency caused by varying padding configurations, we introduce parallel convolutional layers corresponding to different look-ahead settings. Computational latency is controlled through an early-exit mechanism, enabling inference at different network depths. To narrow the performance gap between specialized and flexible models, we propose a two-stage training strategy with a shared-to-multiple decoder transition. Overall, the proposed framework enables a single model to be deployed across diverse latency budgets without retraining separate models.