The paper presents a field study on deploying large inference workloads on a non-GPU AI accelerator and identifies eight categories of limitations.

This paper formalizes token optimization as a multi-objective constrained transformation problem for LLM-based Oracle-to-PostgreSQL migration, demonstrating that adaptive routing offers the best balan…

Taurus is a single-machine system for efficient Graph Neural Network (GNN) inference on large-scale graphs that do not fit in RAM, using source-centric broadcasts and a pipelined GPU-CPU-SSD hierarchy…

This paper proposes a transition-aware backend dispatch approach for efficient large language model inference on edge platforms, reducing latency, energy, and energy-delay product by an average of 17.…

This study benchmarks token-optimized formats (TOON and TRON) against JSON in end-to-end agentic AI systems, finding that TRON significantly reduces token overhead with minimal performance degradation…

This paper introduces a fingerprinting method that exploits subtle numerical deviations in the inference system components (like the engine or hardware) to reliably identify the specific components us…

The paper proposes a federated formal verification architecture that treats verification as a polyglot proof system, successfully validating it on complex production subsystems like a Raft consensus m…

NANOZK introduces a novel, highly efficient zero-knowledge proof system that allows users to cryptographically verify that the output of a large language model (LLM) was generated by a specific, claim…

This paper systematically studies how soft errors propagate during Large Language Model (LLM) inference using a novel fault-injection framework, providing critical insights and mitigation strategies f…

Lodestar is a novel online learning-based request routing system that significantly improves LLM inference efficiency by dynamically assigning incoming requests to the optimal GPU instance to minimize…

The paper identifies a universal, statistically predictable distribution (Mandelbrot) governing LLM outputs, enabling a highly efficient, model-agnostic scoring primitive for provenance and quality as…

The paper demonstrates that for edge-native SLMs used in decentralized governance, simpler, intuitive reasoning (System 1) is significantly more robust and efficient than complex, iterative deliberati…

The paper introduces Opt-Verifier, a novel LLM-based framework that significantly improves the accuracy of automated optimization model generation by implementing dual-side verification from both stru…

This paper proposes Synchronization-Free All-Reduce (SiFAR) to reduce All-Reduce latency and improve end-to-end throughput in low-latency inference systems.

The paper systematically analyzes the benefits and limits of Attention-FFN Disaggregation (AFD) for Mixture-of-Experts (MoE) LLM serving, demonstrating that AFD is crucial for achieving high throughpu…

The paper introduces a lightweight, sampling-based cryptographic protocol for verifiable AI inference that drastically reduces proving overhead from minutes to milliseconds by leveraging statistical p…

The paper presents ExaGEMM, a framework for designing and exploring CPU-native low-bit GEMM via register-resident LUT execution.

SecureRouter is an encrypted routing and inference framework that accelerates secure transformer inference by adaptively selecting the optimal model size based on the encrypted input, achieving a 1.95…

This paper provides the first comprehensive review of threats and defenses specifically targeting on-device AI inference, revealing a significant imbalance where certain attack types, like adversarial…