FMplex: Model Virtualization for Serving Extensible Foundation Models
This paper presents FMplex, a serving system that shares foundation model backbones across multiple tasks to reduce latency and increase task hosting capacity.
FMplex introduces a novel approach to sharing foundation model backbones across multiple tasks, improving performance and efficiency compared to existing methods.
Before reading this…
Applications
- →Language tasks
- →Vision tasks
- →Time-series tasks
- →Multimodal tasks
To understand this paper, make sure you know these concepts first:
- Foundation modelsfind papers →
- Model serving systemsfind papers →
Abstract
More Like ThisFoundation models (FMs) are increasingly used as backbones for downstream tasks across language, vision, time-series, and multimodal applications. Yet existing model-serving systems deploy each customized task as an independent model instance, thereby replicating heavyweight backbones, wasting accelerator memory, and losing opportunities to amortize batching and loading costs. This paper presents FMplex, a serving system that treats FM backbones as a virtualization substrate for deployment sharing. FMplex presents each task with a virtual foundation model (vFM), a logically private FM instance backed by a shared physical FM. This abstraction lets independently customized tasks share a backbone while preserving task-specific extensions, independent lifecycles, and task-level isolation. In addition, we propose a batch-aware fair-queueing scheduler that combines weighted task-level sharing with inter- and intra-task batching across colocated tasks. We implement a FMplex-based serving stack spanning task construction, sharing-aware deployment, and runtime execution. Across 7 FM backbones (16 variants) and 92 downstream tasks, FMplex reduces latency by up to 80% over spatial partitioning and 33.3% over best-effort co-location, while hosting up to 6x more tasks at cluster scale.