The paper introduces CFGzip, an offline token space compression technique that significantly reduces the computational overhead of constrained decoding, making complex grammar enforcement feasible at…

This paper proposes TileLens, a system to mitigate read amplification in Large-Granularity Memory Systems (LGMS) for Large Language Model (LLM) inference by adopting a tile-major layout.

Hawkeye is a system that allows perfect, precision-preserving reproduction of GPU-level matrix multiplication operations on a CPU, enabling efficient and trustworthy third-party auditing of machine le…

The paper introduces $g$MAGNUS, a new algorithm for sparse matrix-matrix multiplication on GPUs that addresses heavy rows by reordering intermediate products and achieves significant speedups.

The paper characterizes 'dead-entry' TLB misses in GPUs, which occur when recently evicted translations are immediately re-walked, and proposes DEPOT, a Bloom filter mechanism that significantly reduc…

This paper introduces a streaming model with catalytic memory and shows dramatic space savings for data stream algorithms, providing exact computation of frequency moments using multi-pass algorithms…

This paper evaluates the performance of large language models on popular benchmarks and finds that only a small percentage of the performant implementations are significantly faster than canonical sol…

The paper proposes EPIC, an efficient and parallel decoding framework that significantly speeds up the process of constraining diffusion language model outputs using Context-Free Grammars (CFG).

This paper proposes a hybrid scheduler that dynamically switches between exclusive batching and mixed batching for LLM inference, achieving superior throughput, especially on bandwidth-constrained GPU…

First implementation of a 3D Gaussian renderer on an Intelligence Processing Unit (IPU) with 1,472 independent tiles using only on-chip SRAM.

This paper presents four novel parallel algorithms for maintaining a rooted spanning forest in dynamic graphs, achieving a throughput of 2 million insertions and 1.4 million deletions per second.

Xvada introduces new techniques for deterministic context-free grammar inference, improving accuracy and compactness over existing approaches, and discovers vulnerabilities in Python Liquid engine.

This paper introduces decode-time grammars to ensure grammatical and semantic correctness of code generated by large language models, particularly for low-resource programming surfaces.

This paper introduces an algorithm for constructing the minimum Wheeler DFA for a given DFA in near-optimal, linearithmic time.