Incast-Free MoE Rate-Based Scheduling
This paper proposes a proactive fair scheduling framework to prevent fabric oversubscription and eliminate incast in Mixture of Experts (MoE) architectures, demonstrating consistent link utilization and reduced Collective Completion Time (CCT) through simulations.
Proposed a proactive fair scheduling framework to prevent fabric oversubscription and eliminate incast in MoE architectures.
Before reading this…
Applications
- →Large language models
To understand this paper, make sure you know these concepts first:
- Understanding of Mixture of Experts architecturesfind papers →
Abstract
More Like ThisMixture of Experts (MoE) architectures have become key to large language models; however, their typical round-robin (RR) scheduling introduces significant bottlenecks. In this paper, we demonstrate that RR causes a previously-undiscovered exponential incast phenomenon with MoE traffic. We propose an alternative proactive fair scheduling framework tailored for MoE workloads, which effectively prevents fabric oversubscription. We also outline how it can be implemented in NICs. Finally, through extensive simulations with real and synthetic workloads, we demonstrate that this framework consistently eliminates incast, maintains a near-100% link utilization, and reduces Collective Completion Time (CCT).