ArXivCSExplorer
☆☆Bookmarks🏆RSSHow to UseFAQ
Built with and by Teycir Ben Soltane•
How to Use•FAQ•GitHub•arXiv.org•
Share:

20 results for “evaluation framework”

CS papers only

Hybrid search: Keyword + semantic, ranked by combined score.ⓘ

Want pure semantic search? Try claim verification →

cs.AIRecentMay 27, 2026

A Unified Framework for the Evaluation of LLM Agentic Capabilities

Pengyu Zhu, Lijun Li, Yaxing Lyu, Qianxin Luo +7 more

The paper introduces a unified framework to fairly evaluate LLM agentic capabilities by standardizing diverse benchmarks and separating the effects of the LLM model from the surrounding framework and…

View →
cs.AIRecentJun 1, 2026

BADGER: Bridging Agentic and Deterministic Evaluation for Generative Enterprise Reasoning

Shannon Serrao, Soumitra Chatterjee, Dorina Strori, Abhishek Sharma +1 more

BADGER is a unified, production-grade evaluation framework that integrates text-to-SQL assessment with agentic behavior evaluation, significantly outperforming existing benchmarks on industry queries.

View →
cs.CLcs.LGEmpiricalRecentJul 30, 2026

(Towards) Scalable Reliable Automated Evaluation with Large Language Models

Bertil Braun, Martin Forell

This paper introduces a novel evaluation framework for Large Language Models using pairwise comparisons and Elo rating system.

View →
cs.AIEmpiricalRecentJun 11, 2026

Evaluation Sovereignty in Metadata-Driven Classification: A Multi-Track Framework for Weakly Supervised Information Systems

Raymond Vasquez

This paper examines the validity of performance measurement in machine learning under different label-authority regimes.

View →
cs.AIRecentMay 29, 2026

LLM-FACETS: A Privacy-Preserving Framework for Evaluating LLM Transparency and Accountability

Tom Lucas, Alessio Buscemi, Alfredo Capozucca, German Castignani +1 more

LLM-FACETS introduces an open-source, privacy-preserving framework designed to enable non-technical domain experts and compliance officers to audit and evaluate the transparency and accountability of…

View →
cs.AIcs.CLcs.CYEmpiricalRecentJun 12, 2026

Every Eval Ever: A Unifying Schema and Community Repository for AI Evaluation Results

Jan Batzner, Sree Harsha Nelaturu, Anastassia Kornilova, Jon Crall +43 more

The paper introduces Every Eval Ever, a shared schema and community-crowdsourced repository for AI evaluation results to standardize and compare AI evaluations.

View →
cs.CLcs.AIEmpiricalRecentAug 10, 2026

From Values to Benchmarks: Evaluating Large Language Models for Governmental Use in Dutch

Laurens Samson, Iva Gornishka, Gossa Lô, Yuki M. Asano +1 more

The paper introduces the 'Grip on LLMs' framework, a systematic evaluation suite for Dutch governmental use of large language models, identifying six evaluation dimensions and operationalising them in…

View →
cs.AIRecentMay 27, 2026

Measuring Progress Toward AGI: A Cognitive Framework

Ryan Burnell, Yumeya Yamamori, Orhan Firat, Kate Olszewska +9 more

The paper introduces a Cognitive Taxonomy and a rigorous evaluation protocol to provide an objective, multi-faceted framework for measuring system capabilities and tracking progress toward Artificial…

View →
cs.AIRecentMay 27, 2026

Benchmarking AI for low-resource contexts: Thinking beyond leaderboards

Aakash Pant, Kavya Shah, Apoorv Agnihotri, Sneha Nikam +2 more

The paper critiques current AI benchmarking practices for low-resource settings, arguing that evaluation must shift focus from isolated model performance to the holistic performance of the deployed sy…

View →
cs.MAEmpiricalRecentJun 15, 2026

Human-on-the-Bridge: Scalable Evaluation for AI Agents

Fouad Bousetouane

This paper introduces Human-on-the-Bridge (HOB), a scalable evaluation paradigm for agentic AI that amplifies evaluation quality without requiring equally large evaluator models.

View →
cs.CRcs.AIcs.CYApr 28, 2026

Making AI-Assisted Grant Evaluation Auditable without Exposing the Model

Kemal Bicakci

The paper proposes a TEE-based architecture that enables external, auditable verification of AI-assisted grant evaluations without exposing the proprietary model, scoring logic, or intermediate reason…

View →
cs.NIcs.CYEmpiricalRecentJun 9, 2026

Internet Quality Barometer (IQB): A preliminary data-driven evaluation of the IQB framework

Pavlos Sermpezis, Zeynep Arslan

The Internet Quality Barometer (IQB) framework maps Internet measurement data to actionable insights about Internet quality.

View →
cs.SEcs.AIcs.CLEmpiricalRecentJun 16, 2026

A Framework for Evaluating Agentic Skills at Scale

Maksim Shaposhnikov, Nicolas Fortuin, Simon Stipcich, Maria I. Gorinova +2 more

The authors present an evaluation framework for agent skills that assesses their utility and provides a reusable methodology for evaluating individual skills.

View →
cs.SEcs.AIRecentJun 3, 2026

From Prompt to Process: a Process Taxonomy and Comparative Assessment of Frameworks Supporting AI Software Development Agents

Sanderson Oliveira de Macedo

This paper studies AI development frameworks for software engineering and proposes a six-dimension process taxonomy.

View →
cs.CRcs.AIcs.CLApr 29, 2026

LATTICE: Evaluating Decision Support Utility of Crypto Agents

Aaron Chan, Tengfei Li, Tianyi Xiao, Angela Chen +2 more

The paper introduces LATTICE, a novel benchmark for evaluating how well crypto agents assist user decision-making, finding that different agents excel in different specific areas rather than having a…

View →
cs.SEcs.AIEmpiricalRecentNov 6, 2026

DRAFT-Bench: Evaluating Frontier Agents Across the Professional Design Lifecycle

Rong Pan, Jianqiang Wang

This paper introduces DRAFT-Bench, a lifecycle-oriented evaluation framework for assessing the capabilities of frontier general-purpose agents in design tasks.

View →
cs.CLcs.AIEmpiricalRecentJun 16, 2026

RubricsTree: Scalable and Evolving Open-Ended Evaluation of Personal Health Agents across Health Memory and Medical Skills

Weizhi Zhang, Zechen Li, Hamid Palangi, Ben Graef +15 more

The authors introduce RubricsTree, a scalable evaluation framework for personal health agents that uses a hierarchical taxonomy of clinically-verifiable rubrics to improve the evaluation of large-scal…

View →
cs.CRMar 23, 2026

Framework for Risk-Based IoT Cybersecurity Audit Engagements

Danielle Hanson, Jeremy Straub

This paper proposes a comprehensive, risk-based auditing framework designed to help internal and external auditors assess the cybersecurity risks posed by diverse IoT devices within corporate and indu…

View →
cs.AIRecentMay 27, 2026

BEAMS: Benchmarking and Evaluating AI for Modeling and Simulation

Sara Metcalf, William Schoenberg

The BEAMS initiative establishes comprehensive benchmarks and evaluates AI tools for modeling and simulation, finding that current AI tools excel at qualitative discussion tasks but struggle with comp…

View →
cs.AIRecentMay 31, 2026

TravelEval: A Comprehensive Benchmarking Framework for Evaluating LLM-Powered Travel Planning Agents

Weiyi Chen, Shuaixiong Wang, Ziyun Gao, Kaichun Hu +4 more

The paper introduces TravelEval, a comprehensive, six-dimensional benchmarking framework that evaluates LLM-powered travel plans using realistic spatio-temporal simulation, revealing that current LLMs…

View →