This paper surveys 257 papers on validating agentic AI systems, identifying mature areas and gaps in behavioral, safety, temporal, regulatory, and multi-agent concerns.
Provides a taxonomy and analysis of validation approaches for agentic AI systems.
Before reading this…
Applications
To understand this paper, make sure you know these concepts first:
Agentic AI systems act through multi-step trajectories that combine planning, tool use, memory, interaction, and adaptation. This behavior stretches validation practice beyond component testing and one-shot input--output evaluation, because acceptable system behavior now depends on how decisions unfold over time and under changing environmental conditions. This survey synthesizes 257 papers spanning agent evaluation, software assurance, cyber-physical systems, runtime monitoring, and regulatory guidance in order to characterize the validation problem for agentic systems. The review is organized around a five-dimension taxonomy covering behavioral, safety, temporal, regulatory, and multi-agent concerns, and uses that taxonomy to map current approaches and expose recurrent coverage gaps. The analysis shows that behavioral evaluation is comparatively mature, while temporal validity, runtime evidence maintenance, regulatory legibility, and open-ended multi-agent systems assurance remain under-developed. Three cross-domain case studies (medical care, industrial operations, smart-mobility systems) provide operational illustrations of how the five taxonomy dimensions recur in safety-critical settings, grounded in the failure patterns documented in the reviewed literature. The paper concludes with a lifecycle-oriented research agenda centered on bounded-autonomy specifications, adversarial trajectory generation, runtime monitoring, and audit-ready evidence structures. The central claim is that trustworthy deployment of agentic AI depends on validating trajectories in context rather than assessing isolated components alone.
Engineering Trustworthy Agentic AI for Critical Systems
This paper surveys the trustworthiness of agentic AI systems in engineering doma…
Evaluating Agentic Bioinformatics through Function, Evidence, and Validation
The paper introduces the Function--Evidence--Validation (FEV) framework to asses…
Agents in the Wild: Where Research Meets Deployment
This tutorial explores advances and challenges in deploying large language model…
Autonomous Repair for Multi-Agent Systems via Monte-Carlo Tree Search
Proposed MARS, a search-based framework for multi-agent system repair using Mont…
ARCHER: Agentic Rule and Compliance Harness for Executable Regulations
ARCHER is a test-driven, multi-agent program-synthesis harness that generates au…
JANUS: Foreseeing Latent Risk for Long-Horizon Agent Safety
The paper proposes Janus, a framework for long-horizon agent safety using multi-…
Deep Agentic Search for Repository-Level Code Question Answering: An Empirical Study
This paper compares Semantic Search and Deep Agentic Search for code question an…
On AI Safety and Security Technical Debt in Engineering AI-Enabled Systems
This paper identifies 31 distinct types of Artificial Intelligence (AI) Technica…