cs.CL
ExpGraph: Model-Agnostic Experience Learning with Graph-Structured Memory for LLM Agents
May 29, 2026
ExpGraph is a model-agnostic framework that uses a self-evolving experience graph to enable LLM agents to reuse past successful strategies and failure lessons, significantly improving performance across diverse tasks.
Large language model (LLM) agents have shown strong capabilities in reasoning, tool use, and multi-step interaction, but they often solve tasks from scratch and fail to reuse successful strategies or failure lessons from prior experience. Fine-tuning on collected experience can improve reuse, but it is inflexible when stronger or more suitable executors emerge. We propose ExpGraph, a model-agnostic experience learning framework that enables frozen and replaceable LLM executors to improve through external experience reuse without parameter updates. ExpGraph summarizes historical trajectories into reusable skills and failure lessons, organizes them as nodes in a self-evolving experience graph, and retrieves useful experiences through graph diffusion and utility-aware ranking. A lightweight retrieval copilot is trained with reinforcement learning using feedback that compares executor performance with and without retrieved experiences, while the graph is updated online from downstream task outcomes. We evaluate ExpGraph on ExpSuite, covering question answering, mathematical reasoning, code generation, and multi-step agentic environments including ALFWorld and AppWorld. ExpGraph improves over the strongest baseline by 12.2% and 4.7% on static tasks with smaller and larger executors, and by 21.4% and 12.7% in agentic environments, while reducing average interaction steps by 12.7% and 21.6%. Ablations show that graph-structured experience, utility-aware ranking, and adaptive retrieval jointly enable effective experience reuse across diverse tasks and executor models.
ExpWeaver: LLM Agents Learn from Experience via Latent RAG
ExpWeaver introduces a novel framework for LLM agents to learn from past experie…
Unified Context Evolution for LLM Agents
The paper introduces Unified Context Evolution (UCE), a gradient-free framework…
Better with Experience: Self-Evolving LLM Agents for Evidence-Grounded Health Community Notes
The paper introduces EvoNote, a self-evolving agentic framework that significant…
Make LLM Learn to Synthesize from Streaming Experiences through Feedback
The paper introduces StreamSynth, a sequential setting for synthetic data genera…
Evolve as a Team: Collaborative Self-Evolution for LLM-based Multi-Agent Systems
The paper proposes Meta-Team, an experience-driven framework that enables multi-…
ElasticMem: Latent Memory as a Learnable Resource for LLM Agents
ElasticMem introduces a novel framework that treats memory as an elastic latent…
EvoDefense: Co-Evolving Black-Box Defense with Large Language Models
EvoDefense introduces an experience-guided, co-evolving black-box defense mechan…
Learning to Retrieve: Dual-Level Long-Term Memory for Text-to-SQL Agents
The paper proposes MERIT, a dual-level, multi-horizon memory retrieval framework…