Hawkeye is a system that allows perfect, precision-preserving reproduction of GPU-level matrix multiplication operations on a CPU, enabling efficient and trustworthy third-party auditing of machine le…

This paper introduces X-Stage, a software-visible post-issue pipeline stage to improve communication efficiency in distributed diffusion transformer (DiT) inference, leading to significant speedups fo…

This paper compares and optimizes CUDA strategies for a shallow neural network, achieving a 1.41x speedup on a large dataset.

The paper introduces Rotary GPU, an exploratory execution approach demonstrating that large Mixture-of-Experts models can be run locally on consumer GPUs with limited VRAM, achieving usable decode thr…

This paper introduces a distribution-aware framework for modeling and benchmarking Mixture-of-Experts (MoE) inference, showing that the best fused-MoE kernel changes with routing skew and token count,…

The paper proposes a fine-grained approach to overlap expert compute with the second all-to-all communication in Mixture-of-Experts (MoE) models, improving distributed MoE execution efficiency on mult…

The paper presents a field study on deploying large inference workloads on a non-GPU AI accelerator and identifies eight categories of limitations.

This paper introduces SwarmX, a system for scheduling agentic AI applications in GPU-CPU clusters using neural predictors, reducing tail latency by up to 61.5% and sustaining up to 2x the throughput o…

TileSight is a tile-centric performance-modeling tool that predicts single-GPU kernel latency and cache hit rates with low error, outperforming state-of-the-art baselines and transferring well across…

This paper presents a memory-efficient training stack for Mixture-of-Experts (MoE) models, combining and specializing parallelism techniques for maximal efficiency.

This paper identifies suboptimalities in default GPU-first scheduling for AI tool workloads and proposes an agentic scheduler that adaptively assigns tools to GPU or CPU based on runtime factors.

This paper introduces the Distributed Truncated Spectral Transform (DTST) for Fourier Neural Operators (FNOs), achieving significant speedups in distributed computing.

Taurus is a single-machine system for efficient Graph Neural Network (GNN) inference on large-scale graphs that do not fit in RAM, using source-centric broadcasts and a pipelined GPU-CPU-SSD hierarchy…

This paper demonstrates that Large Language Models (LLMs) can serve as accurate and selective surrogates for costly GPU kernel performance measurements, significantly expanding the search space for op…

Lodestar is a novel online learning-based request routing system that significantly improves LLM inference efficiency by dynamically assigning incoming requests to the optimal GPU instance to minimize…

This paper proposes FSZ, a GPU error-bounded lossy compressor with three innovations for higher compression ratios and throughput within a single CUDA kernel.