20 results for “Understanding of agent systems and memory benchmarks”
CS papers onlyHybrid search: Keyword + semantic, ranked by combined score.ⓘ
Want pure semantic search? Try claim verification →
Zhishang Xiang, Zerui Chen, Yunbo Tang, Zhimin Wei +4 more
Proposed MemSyco-Bench benchmark for evaluating memory-induced sycophancy in agent systems, measuring when and how valid memories should be used.
This paper introduces the Always-On Evaluation Protocol (AOEP-v0) for evaluating always-on agents by focusing on state mutation and recovery obligations.
Qingshan Liu, Guoqing Wang, Wen Wu, Jingqi Huang +4 more
MemPro introduces a system-level evolution framework that treats the entire memory construction-retrieval pipeline as an evolvable program, significantly improving long-horizon agent performance over…
Pritam Dash, Tongyu Ge, Aditi Jain, Tanmay Shah +1 more
This paper systematically studies memory poisoning attacks in LLM agents, identifying multiple vulnerabilities and proposing a new benchmark to assess the risk.
Mengyuan Li, Lei Gao, Haoxuan Xu, Jiate Li +4 more
The paper proposes an infrastructure, clawgang and meowtrade, to transform private, non-transferable agent memories into verifiable, tradable economic commodities.
Chen Liu, Ling Chen, Hanzhang Zhou, Xu Zhang +6 more
This paper introduces Active Task Driving Memory (ATMem), an actively maintained execution state for mobile GUI agents, and STR-GRPO, an online reinforcement learning method that uses ATMem selectivel…
This survey establishes persistent, writable memory as an independent security problem for LLM agents, proposing a comprehensive framework for 'mnemonic sovereignty' to govern the entire memory lifecy…
This paper analyzes memory poisoning attacks targeting multi-agent systems (MAS) powered by LLMs, proposing mitigation strategies across various memory types, especially focusing on secure design prin…
The paper introduces Portable Agent Memory, an open protocol designed to allow persistent, cryptographically-verified memory state to be reliably transferred between diverse and heterogeneous AI agent…
Nhat Minh Le, Yisen Xu, Zhijie Wang, Tse-Hsun +1 more
This paper evaluates the performance of large language models on popular benchmarks and finds that only a small percentage of the performant implementations are significantly faster than canonical sol…
Weihuang Zheng, Tianyuan Zou, Eileen Ye, Alphet Liu +4 more
The paper introduces E-Bench, a synthetic benchmark for evaluating multi-step tool use in Large Language Models across three product domains.
Chenyu Wang, Zishen Wan, Jeffrey Ma, Shvetank Prakash +7 more
This paper introduces ArchEval, a benchmark and platform for evaluating LLM agents on computer architecture design and optimization.
The paper introduces AGENTCL, a rigorous evaluation framework that uses controlled task streams to accurately measure an agent's ability to accumulate and reuse knowledge across multiple tasks, thereb…
The paper introduces Momento, a new benchmark that evaluates agentic AI's ability to maintain state and reason across multiple, disconnected sessions, revealing that current agents struggle with integ…
The paper introduces memorywire, a vendor-neutral JSON-Schema wire format and reference implementation designed to standardize and govern memory operations across disparate agent-memory frameworks.
The paper proposes Multi-Agent Computer Use (MACU) systems, which significantly improve performance on complex, long-horizon tasks by enabling parallel execution and dynamic task decomposition compare…
This paper proposes an organizational memory for LLM-based agents to access and share enterprise-specific procedural knowledge for reliable business process execution.
Yaxuan Kong, Qingren Yao, Yuqi Nie, Yichen Li +6 more
The paper introduces TimeSage-MT, a comprehensive multi-turn benchmark designed to rigorously test an LLM agent's ability to perform complex, evolving time series analysis, revealing critical gaps in…