20 results for “real-time inference”
CS papers onlyHybrid search: Keyword + semantic, ranked by combined score.ⓘ
Want pure semantic search? Try claim verification →
This paper extends hls4ml to deploy spiking neural networks on FPGA firmware for real-time inference.
The paper analyzes the failure modes of aggressive 2-bit quantization in large reasoning models, proposing lightweight controls like FP16 planning and loop rescue to restore accuracy and achieve pract…
Yuhang Chen, Jinhao Duan, Ruichen Zhang, Mingfu Liang +10 more
This paper proposes Learning to Allocate (L2A), an end-to-end framework for resource-adaptive inference in Large Language Models (LLMs) using budget-conditioned and input-aware gating networks.
Paras Dahal, Anton Bakhtin, Taco Cohen, Zhengxing Chen +8 more
The paper introduces agentic meta-reasoning, a structured reasoning process for controlling the execution of complex tasks.
The paper proposes an architecture for sequential statistical procedures that can be used in various applications.
Dezhi Yi, Huifeng Guo, Kunpeng Xie, Zhaolong Jian +5 more
This paper proposes DPIFrame, a dual parallelizable framework for accelerating Click-through rate (CTR) model inference on GPU, achieving state-of-the-art inference performance with significant speedu…
This paper proposes Chained Recursive Language Models (Chained RLM), an inference-time architecture for managing context in large language models, allowing for intermediate artifacts to be inspected,…
The paper introduces GLIDE, an open-source Python library that unifies multiple state-of-the-art Prediction-Powered Inference (PPI) estimators and samplers to provide reliable, debiased estimates and…
Jalaj Bhandari, Dhawal Gupta, Hongli Zhan, Zhenting Wang +4 more
This paper evaluates a simple inference-time scaling approach for large language model agents on complex tasks.
A hardwired distributed processing unit (HDPU) is designed to make neural decisions at the sensor, reducing inference on the host.
Arkaprava Sinha, Dominick Reilly, Siddharth Krishnan, Hieu Le +1 more
The paper introduces TimeProVe, a cost-efficient framework for temporally grounded reasoning in long videos.
This paper proposes a method to optimize operator order for real-time inference on small devices without extra latency.
Shiguo Lian, Kai Wang, Zhaoxiang Liu, Wen Liu +21 more
This paper proposes a four-layer technical architecture for large model inference optimization, including Multi-model Fusion, Model Optimization, Compute-Model Fusion, and Compute-Network-Model Fusion…