~ similar to 2606.04908· 20 results
Zhihao Xu, Hao Zhong, Zeting Zhou, Yuhang Xu +8 more
This paper proposes Gleam, a framework for efficient GPU sharing across local-area CUDA devices, reducing bandwidth overhead, improving API call latency, and ensuring context consistency.
Haozhe Wang, Lina Chen, Fanqi Wan, Qi Han +4 more
This paper presents Metal Sandbox, a system for efficient on-device inference on macOS that combines restricted file access, native command execution, and command-level scheduling.
Xinwei Qiang, Yifan Hu, Shixuan Sun, Jing Yang +5 more
GF-DiT: A policy-programmable runtime for elastic DiT serving
This paper presents ScaleRAN, a runtime for elastic GPU resource management in AI-RAN, enabling dynamic partitioning of GPU resources between RAN and AI workloads.
Jiasheng Zhou, Longbin Zeng, Clavis Chen, Ruiming Lu +4 more
ARGUS is a low-overhead, fine-grained, always-on tracing and real-time analysis system for training workloads in large production clusters.
Sookyung Choi, Seungyong Lee, Kangkyu Park, Yunseo Chun +10 more
This paper presents NELSSA, a serving system that integrates GPUs with Processing-near-Memory (PNM) devices to efficiently handle mixed-length workloads in LLMs, achieving up to 5.5x decode throughput…
This paper implements and evaluates Whisper dot-product kernel offloading on a programmable Coarse-Grained Linear Arrays (CGLAs) architecture to reduce power consumption.
The paper characterizes 'dead-entry' TLB misses in GPUs, which occur when recently evicted translations are immediately re-walked, and proposes DEPOT, a Bloom filter mechanism that significantly reduc…
Hawkeye is a system that allows perfect, precision-preserving reproduction of GPU-level matrix multiplication operations on a CPU, enabling efficient and trustworthy third-party auditing of machine le…
Supun Kuruppu, Pranav Dangi, Wei Li, Zhenyu Bai +1 more
This paper improves kernel performance on GPUs by accounting for physical non-idealities in hardware resources.
This paper analyzes data access patterns in large language model workloads on multi-partition GPUs and presents optimization strategies based on inter-workgroup sharing patterns.
Shuheng Gao, Fan Guo, Yongkun Li, Ziye Yue +2 more
gKSM is a transparent GPU memory deduplication framework for serverless LLM inference that reduces GPU memory usage and improves throughput.
This paper provides a workload-grounded characterization study on the usage of the Tensor Memory Accelerator (TMA) on NVIDIA Hopper GPUs.
This paper presents a rustc-based GPU compilation framework for Rust, ensuring memory safety and efficiency in high-performance GPU programming.