ArXivCSExplorer
☆☆Bookmarks🏆RSSHow to UseFAQ
Built with and by Teycir Ben Soltane•
How to Use•FAQ•GitHub•arXiv.org•
Share:

20 results for “Understanding of Mixture-of-Experts models, Distributed systems, Edge computing, Queueing theory”

CS papers only

Hybrid search: Keyword + semantic, ranked by combined score.ⓘ

Want pure semantic search? Try claim verification →

cs.LGcs.AIEmpiricalRecentOct 15, 2026

Rethinking Routing Exploration in Modern Mixture-of-Experts

Bingcong Li, Hao Ma, Michael Muehlebach, Niao He +1 more

This paper proposes MirrorMoE to reconcile the exploration dilemma in mixture-of-experts models by shrinking the exploration space through token-adaptive suppression of unnecessary experts.

View →
math.DScs.AIcs.LGRecentMay 27, 2026

A Minimal Bifurcation Model of Load Imbalance in a Softmax Mixture-of-Experts Router

O. M. Kiselev

The paper develops a minimal dynamical model showing that adaptive softmax routing in Mixture-of-Experts (MoE) layers can undergo abrupt transitions to load imbalance via bifurcation mechanisms.

View →
cs.DCcs.AIEmpiricalRecentJul 2, 2026

Mixture-of-Parallelisms: Towards Memory-Efficient Training Stack for Mixture-of-Experts Models

Xuan-Phi Nguyen, Shrey Pandit, Yiran Zhao, Semih Yavuz +2 more

This paper presents a memory-efficient training stack for Mixture-of-Experts (MoE) models, combining and specializing parallelism techniques for maximal efficiency.

View →
cs.DCEmpiricalRecentOct 26, 2026

INC-MoE: Overlapping In-Network Collectives and Expert Computation for Expert-Parallel Mixture-of-Experts Inference

John Stephan, Razvan Puscasu, Xu Yong, Ding Jiutao +1 more

This paper presents INC-MoE, an expert-parallel MoE layer that offloads dispatch and combine operations to the network switch to reduce latency and increase throughput.

View →
cs.LGcs.AIRecentJun 1, 2026

ProbMoE: Differentiable Probabilistic Routing for Mixture-of-Experts

Heng Zhao, Zilei Shao, Guy Van den Broeck, Zhe Zeng

The paper introduces ProbMoE, a probabilistic routing framework that tackles the non-differentiability of top-$k$ routing in Mixture-of-Experts (MoE) models, achieving strong performance with improved…

View →
cs.AIRecentMay 31, 2026

DAG-MoE: From Simple Mixture to Structural Aggregation in Mixture-of-Experts

Jiarui Feng, Hanqing Zeng, Karish Grover, Ruizhong Qiu +10 more

The paper proposes DAG-MoE, a novel sparse Mixture-of-Experts framework that replaces standard weighted-sum aggregation with structural aggregation to enhance model performance and enable multi-step r…

View →
cs.LGcs.AIcs.DCEmpiricalRecentJun 17, 2026

FoMoE: Breaking the Full-Replica Barrier with a Federation of MoEs

Lorenzo Sani, Zeyu Cao, Meghdad Kurmanji, Alex Iacob +4 more

This paper introduces FoMoE, a system that reduces communication costs and achieves speedups in training large language models by partitioning expert layers across workers.

View →
cs.DCcs.AIEmpiricalRecentJul 21, 2026

Fine-grained Computation-Communication Overlap via Tile-level Signaling and Scheduling for Mixture-of-Experts

Minyu Cui, Anna Wingkvist, Morgan Ericsson

The paper proposes a fine-grained approach to overlap expert compute with the second all-to-all communication in Mixture-of-Experts (MoE) models, improving distributed MoE execution efficiency on mult…

View →
cs.LGcs.AIcs.CLEmpiricalRecentJun 10, 2026

Redesign Mixture-of-Experts Routers with Manifold Power Iteration

Songhao Wu, Ang Lv, Ruobing Xie, Yankai Lin

This paper proposes a new router redesign for Mixture-of-Experts models using Manifold Power Iteration to align router rows with the principal singular directions of associated experts.

View →
cs.DCcs.LGEmpiricalRecentOct 13, 2026

FastEEP: Fast Intra-Instance Elastic Scaling for Expert-Parallel MoE Serving

Yunfei Feng, Jingkai He, Shiqi Liu, Cheng Zhang +2 more

The paper presents FastEEP, a system for fast intra-instance scaling of Mixture-of-Experts (MoE) models in production.

View →
cs.LGcs.AIRecentMay 31, 2026

Beyond Task-Agnostic: Task-Aware Grouping for Communication-Efficient Multi-Task MoE Inference

Zhiyao Xu, Aoxue Liu, Zhanjie Ding, Dan Zhao +2 more

The paper proposes Task-Aware Coactivation Grouping (TACG) to significantly reduce communication costs in multi-task MoE inference by grouping experts based on task-specific co-activation patterns, ou…

View →
cs.AImath.OCRecentJun 1, 2026

Stochastic convergence of parallel asynchronous adaptive first-order methods

Serge Gratton, Philippe L. Toint

The paper analyzes a new class of asynchronous adaptive first-order optimization methods and proves their stochastic convergence rate is O(1/sqrt{t}) for non-convex functions.

View →
cs.LGcs.AIRecentMay 28, 2026

Graph-Conditioned Mixture of Graph Neural Network Experts for Traffic Forecasting

Amirhossein Ghaffari, Saeid Sheikhi, Ekaterina Gilman

The paper proposes GC-MoE, a graph-conditioned Mixture of Experts framework, to improve traffic forecasting by assigning personalized, specialized forecasting experts to individual road segments.

View →
cs.CRcs.AIEmpiricalRecentJun 29, 2026

A Multi-task Mixture of Experts Framework for Malware Classification, Packing Detection, and Family Attribution

Jithin S., Roshin Sleeba C., Anvin Mariya P. B., Asmitha K. A. +3 more

A unified multi-task malware analysis framework based on Mixture of Experts (MoE) architectures is proposed for malware family classification, packed versus unpacked detection, and malware versus beni…

View →
cs.NIcs.CLEmpiricalRecentAug 1, 2026

HetRoute Heterogeneous and Cost-aware Collaborative Routing Framework for Distributed Edge MoE Inference

Xin Yuan, Ning Li, Wenchao Xu, Athanasios V. Vasilakos +2 more

This paper proposes HetRoute, a framework for optimizing Mixture-of-Experts model inference over geo-distributed heterogeneous edge servers by considering jointly the effects of cross-server link band…

View →
cs.NIcs.DCcs.LGEmpiricalRecentJul 28, 2026

Incast-Free MoE Rate-Based Scheduling

Evyatar Cohen, Jose Yallouz, Alexander Shpiner, Mark Silberstein +2 more

This paper proposes a proactive fair scheduling framework to prevent fabric oversubscription and eliminate incast in Mixture of Experts (MoE) architectures, demonstrating consistent link utilization a…

View →
cs.DCEmpiricalRecentSep 18, 2026

Weave: Fine-Grained Dynamic SM Scheduling in an MoE Megakernel for Compute-Communication Overlap

Ziyu Huang, Yangjie Zhou, Chenhao Zhu, Zihan Liu +12 more

This paper presents Weave, a system that improves the performance of Mixture-of-Experts (MoE) inference by dynamically scheduling GPU resources.

View →
cs.AREmpiricalRecentJul 25, 2026

Decoding the Skew: Distribution-Aware MoE Inference with Adaptive Kernel Dispatch

En-Ming Huang, An-Cheng Chang, Bai-Cheng Jeng, Shih-Hao Hung +1 more

This paper introduces a distribution-aware framework for modeling and benchmarking Mixture-of-Experts (MoE) inference, showing that the best fused-MoE kernel changes with routing skew and token count,…

View →