20 results for “key-value cache”
CS papers onlyHybrid search: Keyword + semantic, ranked by combined score.ⓘ
Want pure semantic search? Try claim verification →
The paper proposes adaptive caching for key-value (KV) caching in large language models, improving cache hit rate and reducing time to first token.
This paper introduces HIJACKKV, the first attack framework to exploit the new threat of KV Cache Hijacking in large language models, achieving an average success rate of 94%.
Junjie Peng, You Wu, Haoyi Wu, Jialong Han +3 more
GRKV introduces a training-free KV-cache merging method that uses global regression to distribute information from evicted tokens, solving the over-merging problem inherent in span-based retention.
Guanlong Wu, Zhaohan li, Yao Zhang, Zheng Zhang +3 more
CachePrune introduces a privacy-aware, fine-grained KV cache sharing mechanism that allows LLM inference systems to safely reuse cache entries across users' requests, significantly improving efficienc…
Chunan Shi, Yilei Chen, Yilin Chen, Xupeng Miao +1 more
The paper proposes AsymCache, a computation-latency-aware KV cache management system that optimizes LLM inference by aligning cache eviction decisions with GPU attention kernel performance, significan…
This paper presents ZipFusion, a method that enables attention to consume losslessly compressed key-value (KV) cache directly, improving model decode throughput and reducing memory-bandwidth bound.
Jinnan Yang, Yan Wang, Zhen Bi, Kehao Wu +4 more
WaveFilter is a novel, training-free framework that uses wavelet transforms to efficiently filter critical tokens in the KV cache, significantly improving the long-context performance of Diffusion LLM…
Zaifeng Pan, Lucas Wang, Xinwei Qiang, Yanbo Zhou +4 more
This paper presents HiState, a prefix cache management system for hybrid models serving agent workloads to reduce inference costs.
The paper proposes GroundedCache, an evidence-validated cache router that significantly improves the safety of reusing cached semantic answers in RAG systems by requiring multiple gates to validate th…
This paper proposes C^2, a set of techniques to improve the spatial locality of succinct tries, resulting in faster query performance and reduced memory footprint.
This paper introduces new metrics to evaluate semantic caching systems and shows that model selection is a calibration problem, not a ranking one.
Shubham Tiwari, Tapan Chugh, Nash Rickert, Simon Peter +2 more
This paper presents a new KVCache management layer called CacheWise that improves performance for coding agent workloads.
This paper introduces Foresight, a cache-friendly skiplist optimization that improves performance in sequential and concurrent settings.
Mufei Li, Shikun Liu, Dongqi Fu, Haoyu Wang +4 more
The paper presents KVEraser, a learned KV-cache editing method for efficient localized context erasing in long-context LLM applications.
The paper characterizes the memory-rate trade-off for demand-private coded caching with heterogeneous file sizes and cache capacities.
Proposed a novel algorithm-hardware co-design, HiKV, to tackle the memory bottleneck in long-context large language models by exploiting KV cache redundancy through hierarchical importance awareness,…
This paper introduces DiStash, a disaggregated transactional key-value store that enables an application to use a single transaction to manage key-value pairs across different pools of stashes, preven…