ArXivCSExplorer
☆☆Bookmarks🏆RSSHow to UseFAQ
Built with and by Teycir Ben Soltane•
How to Use•FAQ•GitHub•arXiv.org•
Share:

20 results for “evaluation protocol”

CS papers only

Hybrid search: Keyword + semantic, ranked by combined score.ⓘ

Want pure semantic search? Try claim verification →

cs.AIcs.CRMay 11, 2026

From Controlled to the Wild: Evaluation of Pentesting Agents for the Real-World

Pedro Conde, Henrique Branquinho, Valerio Mazzone, Bruno Mendes +2 more

The paper introduces a novel, practical evaluation protocol that shifts the assessment of AI pentesting agents from simple task completion to validated, open-ended vulnerability discovery in complex,…

View →
cs.LGcs.AIRecentMay 27, 2026

FormInv: A Measurement Protocol for Semantic Invariance in Mathematical Reasoning Benchmarks

Nishal Thomas, Noel Thomas

The paper introduces FormInv, a measurement protocol that reveals significant semantic inconsistencies in existing mathematical reasoning benchmarks, showing that standard accuracy metrics fail to cap…

View →
cs.CRMar 25, 2026

AgentRFC: Security Design Principles and Conformance Testing for Agent Protocols

Shenghan Zheng, Qifan Zhang

The paper introduces a comprehensive security framework, AgentRFC, to systematically analyze and test the security conformance of various AI agent protocols, identifying critical design gaps, especial…

View →
cs.CRcs.AIcs.LGMay 11, 2026

Acceptance Cards:A Four-Diagnostic Standard for Safe Fine-Tuning Defense Claims

Phongsakon Mark Konrad, Toygar Tanyel, Serkan Ayvaz

The paper introduces Acceptance Cards, a rigorous four-diagnostic standard, to provide a comprehensive and reliable evaluation protocol for claims of safe fine-tuning defenses.

View →
cs.AIcs.CLcs.CYEmpiricalRecentJun 12, 2026

Every Eval Ever: A Unifying Schema and Community Repository for AI Evaluation Results

Jan Batzner, Sree Harsha Nelaturu, Anastassia Kornilova, Jon Crall +43 more

The paper introduces Every Eval Ever, a shared schema and community-crowdsourced repository for AI evaluation results to standardize and compare AI evaluations.

View →
cs.CRcs.AIcs.CYApr 28, 2026

Making AI-Assisted Grant Evaluation Auditable without Exposing the Model

Kemal Bicakci

The paper proposes a TEE-based architecture that enables external, auditable verification of AI-assisted grant evaluations without exposing the proprietary model, scoring logic, or intermediate reason…

View →
cs.MAEmpiricalRecentJun 15, 2026

Human-on-the-Bridge: Scalable Evaluation for AI Agents

Fouad Bousetouane

This paper introduces Human-on-the-Bridge (HOB), a scalable evaluation paradigm for agentic AI that amplifies evaluation quality without requiring equally large evaluator models.

View →
cs.CRcs.AIcs.SERecentMay 22, 2026

Attested Tool-Server Admission: A Security Extension to the Model Context Protocol

Alfredo Metere

The paper introduces mcp-attested, a security extension to the Model Context Protocol (MCP) that allows hosts to safely admit and restrict the tools used by external, third-party tool servers.

View →
cs.AIRecentMay 27, 2026

Benchmarking AI for low-resource contexts: Thinking beyond leaderboards

Aakash Pant, Kavya Shah, Apoorv Agnihotri, Sneha Nikam +2 more

The paper critiques current AI benchmarking practices for low-resource settings, arguing that evaluation must shift focus from isolated model performance to the holistic performance of the deployed sy…

View →
cs.AIEmpiricalRecentJun 11, 2026

Evaluation Sovereignty in Metadata-Driven Classification: A Multi-Track Framework for Weakly Supervised Information Systems

Raymond Vasquez

This paper examines the validity of performance measurement in machine learning under different label-authority regimes.

View →
cs.CLcs.AIcs.IREmpiricalRecentOct 15, 2026

AuditKIT: A Unified Evaluation Spine for Benchmarking, Judging, and Comparing Large Language Models and Agentic Systems

Shreeyans Arora, Utsav Avaiya, Zera Lyngkhoi, Hem Gosalia +3 more

The paper presents AuditKIT, a library for evaluating language models and agents by recording various measurements in one result representation.

View →
cs.CRcs.AIcs.LGApr 7, 2026

WebSP-Eval: Evaluating Web Agents on Website Security and Privacy Tasks

Guruprasad Viswanathan Ramesh, Asmit Nayak, Basieem Siddique, Kassem Fawaz

The paper introduces WebSP-Eval, a new framework to evaluate web agents on complex website security and privacy tasks, finding that current state-of-the-art models struggle significantly with stateful…

View →
cs.CRcs.MAApr 6, 2026

HDP: A Lightweight Cryptographic Protocol for Human Delegation Provenance in Agentic AI Systems

Asiri Dalugoda

The paper introduces the Human Delegation Provenance (HDP) protocol, a lightweight, token-based cryptographic scheme designed to verify the full, multi-hop chain of human authorization for actions exe…

View →
cs.CRcs.AIEmpiricalRecentJun 27, 2026

From Tool Connection to Execution Control: Benchmarking Security Invariants in MCP-Style Agent Runtimes

Ting Liu

This paper proposes HCP, a Handle-Capability Protocol runtime for MCP-style agent systems, which implements eight execution-control invariants to enhance security while preserving MCP-like workflows.

View →
cs.AIRecentMay 29, 2026

PReMISE: Policy Rubrics as Measurement Specifications for LLM Judges

Swastik Roy, Rajkumar Pujari, Tharindu Kumarage, Charith Peris +4 more

PReMISE introduces a framework to audit and improve the quality of rubrics used to guide LLM judges, demonstrating that it can significantly increase judge accuracy and reduce the exploitability of re…

View →
cs.CLcs.AIRecentMay 27, 2026

Let the Results Speak: A Replication-First Paradigm for LLM Behavioral Benchmarking

Yuming, Huang, Yao Liu, Lei Wang +1 more

The paper introduces a 'replication-first' paradigm for LLM behavioral benchmarking, demonstrating that this rigorous approach uncovers significant, non-obvious performance drops between successive mo…

View →
cs.SEEmpiricalRecentJun 24, 2026

CodeChat-Eval: Evaluating Large Language Models in Multi-Turn Code Refinement Dialogues

Guoxiang, Guo, Kla Tantithamthavorn, Neelofar Neelofar +2 more

This paper introduces CodeChat-Eval, an evaluation framework for assessing the functional correctness of Large Language Models in multi-turn code refinement dialogues.

View →