cs.IRcs.AIcs.CLcs.LGEmpirical
Knowledge Graph Enhanced Memory-Augmented Retrieval for Long Context Modeling
Jun 12, 2026
This paper introduces KGERMAR, a framework that constructs dynamic, context-specific knowledge graphs during inference for long-context language modeling, achieving lower perplexity and better memory efficiency than memory-augmented baselines.
The paper proposes a dynamic knowledge graph construction approach that adapts to input contexts, unlike memory-augmented language models that rely on fixed knowledge bases.
Keywords
Before reading this…
Applications
To understand this paper, make sure you know these concepts first:
Long-context language modeling requires not only extending context windows but maintaining coherent understanding of entity states and relationships across thousands of tokens -- a challenge that semantic similarity alone cannot address. KGERMAR addresses this by constructing dynamic, context-specific knowledge graphs from input text during inference, enabling domain-adaptive retrieval that leverages both semantic similarity and explicit entity relationships. The framework performs real-time entity and relation extraction to build contextual knowledge graphs, then integrates graph-structural embeddings with textual semantics through a multi-component memory architecture. Three memory banks -- contextual, semantic, and structural -- are maintained with retrieval signals fused via learned weights to capture both surface-level semantics and deeper relational patterns. Evaluated on SlimPajama (84.7K training examples), WikiText-103 (4,358 examples), PG-19 (100 examples), and Proof-pile (46.3K examples), KGERMAR achieves up to 8.5\% lower perplexity and 2--2.5x better memory efficiency than memory-augmented baselines across context lengths from 1K to 32K tokens, with superior in-context learning performance across five NLU tasks. The dynamic knowledge graph construction approach advances memory-augmented language modeling by enabling domain-specific knowledge representation that adapts to input contexts rather than relying on fixed knowledge bases.
Memory poisoning and secure multi-agent systems
This paper analyzes memory poisoning attacks targeting multi-agent systems (MAS)…
Finding Memory Leaks in C/C++ Programs via Neuro-Symbolic Augmented Static Analysis
MemHint is a neuro-symbolic static analysis pipeline that significantly improves…
Opal: Private Memory for Personal AI
Opal is a private memory system for personal AI that maintains high retrieval ac…
Towards Secure Retrieval-Augmented Generation: A Comprehensive Review of Threats, Defenses and Bench…
This paper provides the first comprehensive, end-to-end survey dedicated to the…
Towards Unsupervised Adversarial Document Detection in Retrieval Augmented Generation Systems
The paper proposes an unsupervised method using multiple statistical indicators…
Poison Once, Exploit Forever: Environment-Injected Memory Poisoning Attacks on Web Agents
The paper introduces eTAMP, a novel attack that poisons LLM web agents' memory u…
Walma: Learning to See Memory Corruption in WebAssembly
Walma is a machine learning framework that uses memory snapshot classification t…
PIDP-Attack: Combining Prompt Injection with Database Poisoning Attacks on Retrieval-Augmented Gener…
The paper introduces PIDP-Attack, a novel compound adversarial attack that combi…