ArXivCSExplorer
β˜†β˜†BookmarksπŸ†RSSHow to UseFAQ
Built with and by Teycir Ben Soltaneβ€’
How to Useβ€’FAQβ€’GitHubβ€’arXiv.orgβ€’
Share:

20 results for β€œserving system”

CS papers only

Hybrid search: Keyword + semantic, ranked by combined score.β“˜

Want pure semantic search? Try claim verification β†’

cs.DCEmpiricalRecentJun 16, 2026

LUMEN: Coordinated Failure Recovery for Distributed LLM Serving

Zhang Cao, Shujie Han, Juncheng Zhang, Yuanming Ren +2 more

This paper presents LUMEN, a fault-tolerant LLM serving system that improves serving and recovery times by treating recovery as a load-aware coordination problem.

View β†’
cs.DCcs.LGEmpiricalRecentOct 13, 2026

FastEEP: Fast Intra-Instance Elastic Scaling for Expert-Parallel MoE Serving

Yunfei Feng, Jingkai He, Shiqi Liu, Cheng Zhang +2 more

The paper presents FastEEP, a system for fast intra-instance scaling of Mixture-of-Experts (MoE) models in production.

View β†’
cs.DCcs.AIcs.LGEmpiricalRecentJun 8, 2026

FMplex: Model Virtualization for Serving Extensible Foundation Models

Hetvi Shastri, Pragya Sharma, Walid A. Hanafy, David Irwin +2 more

This paper presents FMplex, a serving system that shares foundation model backbones across multiple tasks to reduce latency and increase task hosting capacity.

View β†’
cs.AREmpiricalRecentJul 29, 2026

NELSSA: A GPU-PNM Heterogeneous System for Mixed-Length LLM Serving via Length-based Request Placement

Sookyung Choi, Seungyong Lee, Kangkyu Park, Yunseo Chun +10 more

This paper presents NELSSA, a serving system that integrates GPUs with Processing-near-Memory (PNM) devices to efficiently handle mixed-length workloads in LLMs, achieving up to 5.5x decode throughput…

View β†’
cs.DCEmpiricalRecentJun 17, 2026

TurboServe: Serving Streaming Video Generation Efficiently and Economically

Youhe Jiang, Haoxu Wang, Haotong Bao, Kai Jiang +4 more

TurboServe is a serving system designed for streaming video generation workloads that reduces latency and cost.

View β†’
cs.OScs.DCcs.LGEmpiricalRecentAug 16, 2026

Global Simulation-Guided Dynamic Operator Scheduling for Efficient Multi-Tenant Model Serving

Weinan Liu, Zeyuan Ding, Dian Ding, Chengcheng Wan +4 more

This paper presents SliceScheduler, a dynamic operator-level scheduling system for multi-tenant model serving that improves GPU utilization and maintains SLA.

View β†’
cs.OScs.AIcs.DCEmpiricalRecentAug 15, 2026

From LLM Inference to Agentic Workloads: Characterization and Implications for Serving Systems

Chaokun Chang, Yukun Zhou, Kaihua Fu, Dakai An +9 more

This paper introduces AgentSysBench, a benchmark suite and measurement toolkit for agentic applications, and identifies six properties that distinguish agentic workloads from conventional LLM serving.

View β†’
cs.DCEmpiricalRecentOct 13, 2026

Quota-Based Aging for Prediction-Guided Scheduling in LLM Serving

Hanzhi Zhang, Anxuan Che, Hang Qi, Qiyue Li +1 more

One clear sentence describing what this paper does

View β†’
cs.DCEmpiricalRecentJun 29, 2026

Spandana: Reconciling Strict SLOs with Low Cost under Fine-Grained Load Fluctuations

Dilina Dehigama, Shyam Jesalpura, Zeyu Xu, Marton Nemeth +3 more

The paper introduces Spandana, an architecture that decouples SLO enforcement from cost optimization in cloud-based online services, achieving high utilization, strict SLO adherence, and cost savings.

View β†’
cs.DCcs.MATheoreticalRecentJul 3, 2026

A Workflow-Aware Serving Layer for Agentic Applications

Jiayi Qian, Zishen Wan, Hanchen Yang, Chun Tao +2 more

The paper presents Dyserve, a workflow-aware serving layer for agentic AI applications that compiles per-node model and verifier choices into an integer linear program, allowing for efficient model se…

View β†’
cs.DSTheoreticalRecentJun 19, 2026

Online Stacking with a Few Load/Unload Points

Martin Olsen

A simple online algorithm is presented for the stacking problem to avoid shifts with a sufficient condition involving stacking area dimension, load/unload points, and maximum items.

View β†’
cs.SEcs.AIEmpiricalRecentJun 10, 2026

Characterizing Software Aging in GPU-Based LLM Serving Systems

Domenico Cotroneo, Bojan Cukic

This paper studies software aging in GPU-based LLM serving systems using a 216-hour campaign.

View β†’
cs.ITmath.PRTheoreticalRecentOct 9, 2026

Fundamental Limits of Covert queueing under Busy-Period Observations

Anumula Venkata Sai Sree Sahithi, Arti Yardi, Jithin Ravi

This paper studies the maximum number of jobs a queueing system can serve covertly without being detected.

View β†’
cs.IRcs.AIEmpiricalRecentJul 28, 2026

MARS: Multi-Agent Re-ranking for Repeat-Order Food Delivery Recommendation

Jiahao Tian, Zhenkai Wang

This paper introduces MARS, a framework for repeat-order food delivery recommendation that combines pre-trained language models with collaborative retrieval and contextual filtering.

View β†’