This survey systematically reviews resource consumption threats in large language models (LLMs) to provide a unified view of the problem landscape, from threat induction to mitigation.

This paper discusses the current understanding of Large Language Models (LLMs), their capabilities, and their relationship to human cognition, with a focus on emerging capabilities and mechanistic imp…

The paper quantifies the exact parametric memory capacity of LLMs using LoRA and proposes a new optimization strategy, MemFT, to enhance memory fidelity.

This paper compares the preservation of linguistic content in different text encodings (tokens, bytes, pixels) using a shared bottleneck, revealing their distinct strengths in surface form preservatio…

This paper presents a comprehensive measurement study on the performance and energy consumption of large language models on mobile devices, using five frameworks and three hardware backends, and intro…

This paper introduces BigPower, a hierarchical source-level surrogate model for fine-grained module-level power estimation during CPU design using large language models and architectural hierarchy.

The paper proposes adaptive caching for key-value (KV) caching in large language models, improving cache hit rate and reducing time to first token.

This paper investigates the energy efficiency of Large Language Model (LLM) serving using emerging memory technologies, specifically monolithic 3D (M3D) integration, and presents simulation results sh…

The paper introduces VikingMem, a novel Memory Base Management System that effectively manages the persistent state of long-term LLM interactions by selectively extracting, evolving, and compressing m…

This paper proposes a new architecture for transformer-based language models called 'former', which allocates capacity non-uniformly across network depth by maintaining wider early and late layers whi…

MemGuard introduces a type-aware memory framework to prevent heterogeneous memory contamination in long-term memory-augmented LLMs, significantly improving memory reliability and efficiency.

This paper proposes a context engineering approach that scales memory with distinct information instead of tokens, using a novelty-gated cache and state-space summary.

Proposed a novel algorithm-hardware co-design, HiKV, to tackle the memory bottleneck in long-context large language models by exploiting KV cache redundancy through hierarchical importance awareness,…

The paper introduces RUMBA, a new benchmark for long-term conversational memory in LLMs with a fine-grained taxonomy and unified methodology.

The paper introduces MemTrace, a framework that treats LLM memory pipelines as traceable graphs to systematically diagnose and automatically correct memory-related errors, boosting performance by up t…

This paper proposes a sparse cache with a novel allocation rule based on DP-means clustering for deep recurrent models, achieving full associative recall with only distinct items, outperforming other…

The paper proposes AsymCache, a computation-latency-aware KV cache management system that optimizes LLM inference by aligning cache eviction decisions with GPU attention kernel performance, significan…

The paper introduces Sparse Memory-Efficient Training (SMET), a method that stabilizes and optimizes Dynamic Sparse Training (DST) for large language models, enabling stable and memory-efficient spars…