TileSight is a tile-centric performance-modeling tool that predicts single-GPU kernel latency and cache hit rates with low error, outperforming state-of-the-art baselines and transferring well across…

This paper proposes CrossPool, a serving engine for cold Machine Learning Models (LLMs) that separates weights and KV-cache into two GPU memory pools to improve GPU memory utilization and long-context…

Hawkeye is a system that allows perfect, precision-preserving reproduction of GPU-level matrix multiplication operations on a CPU, enabling efficient and trustworthy third-party auditing of machine le…

This paper introduces SwarmX, a system for scheduling agentic AI applications in GPU-CPU clusters using neural predictors, reducing tail latency by up to 61.5% and sustaining up to 2x the throughput o…

This paper presents Roomie, an architecture for orchestrating model serving in multi-model scenarios on GPUs, reducing SLO violations by predicting and avoiding kernel-level interference.

This paper identifies suboptimalities in default GPU-first scheduling for AI tool workloads and proposes an agentic scheduler that adaptively assigns tools to GPU or CPU based on runtime factors.

This paper presents Festina, a profiling-guided, power-aware control plane for minimizing energy consumption in serverless large language model (LLM) serving.

The paper introduces Rotary GPU, an exploratory execution approach demonstrating that large Mixture-of-Experts models can be run locally on consumer GPUs with limited VRAM, achieving usable decode thr…

The paper proposes a fine-grained approach to overlap expert compute with the second all-to-all communication in Mixture-of-Experts (MoE) models, improving distributed MoE execution efficiency on mult…

The paper proposes Task-Aware Coactivation Grouping (TACG) to significantly reduce communication costs in multi-task MoE inference by grouping experts based on task-specific co-activation patterns, ou…

This paper explores methods to reduce latency in GPU collective communications for large language model inference, achieving near-optimal designs with barrier-free synchronization and efficient use of…

This paper compares and optimizes CUDA strategies for a shallow neural network, achieving a 1.41x speedup on a large dataset.

The paper introduces PRISM, an evaluation framework for assessing POSIX storage systems for AI research workloads, and compares Lustre and NFS based systems.

Taurus is a single-machine system for efficient Graph Neural Network (GNN) inference on large-scale graphs that do not fit in RAM, using source-centric broadcasts and a pipelined GPU-CPU-SSD hierarchy…

This paper proposes TileLens, a system to mitigate read amplification in Large-Granularity Memory Systems (LGMS) for Large Language Model (LLM) inference by adopting a tile-major layout.

This paper introduces DGNA, a methodology to unveil the Non-Uniform Memory Access (NUMA) architecture of GPU memory hierarchy through microbenchmarking and data analysis.

The paper presents CHASE, an application-driven framework that explores physically feasible Cross-layer Heterogeneous System architectures for executing workloads with diverse requirements.

First implementation of a 3D Gaussian renderer on an Intelligence Processing Unit (IPU) with 1,472 independent tiles using only on-chip SRAM.

The paper characterizes 'dead-entry' TLB misses in GPUs, which occur when recently evicted translations are immediately re-walked, and proposes DEPOT, a Bloom filter mechanism that significantly reduc…