ArXivCSExplorer
☆☆Bookmarks🏆RSSHow to UseFAQ
Built with and by Teycir Ben Soltane•
How to Use•FAQ•GitHub•arXiv.org•
Share:

20 results for “evaluation”

CS papers only

Hybrid search: Keyword + semantic, ranked by combined score.ⓘ

Want pure semantic search? Try claim verification →

cs.AIcs.CLcs.CYEmpiricalRecentJun 12, 2026

Every Eval Ever: A Unifying Schema and Community Repository for AI Evaluation Results

Jan Batzner, Sree Harsha Nelaturu, Anastassia Kornilova, Jon Crall +43 more

The paper introduces Every Eval Ever, a shared schema and community-crowdsourced repository for AI evaluation results to standardize and compare AI evaluations.

View →
cs.CRcs.AIcs.CYApr 28, 2026

Making AI-Assisted Grant Evaluation Auditable without Exposing the Model

Kemal Bicakci

The paper proposes a TEE-based architecture that enables external, auditable verification of AI-assisted grant evaluations without exposing the proprietary model, scoring logic, or intermediate reason…

View →
cs.CLcs.AIEmpiricalRecentJun 26, 2026

Can LLMs Judge Better Than They Generate? Evaluating Task Asymmetry, Mechanistic Interpretability and Transferability for In-Context QA

Sambaran Bandyopadhyay

This paper tests the assumption that evaluation is easier than generation in LLM-as-a-Judge and self-evaluation pipelines using a controlled in-context QA setting and reveals that evaluation attends t…

View →
cs.AIEmpiricalRecentJun 11, 2026

Evaluation Sovereignty in Metadata-Driven Classification: A Multi-Track Framework for Weakly Supervised Information Systems

Raymond Vasquez

This paper examines the validity of performance measurement in machine learning under different label-authority regimes.

View →
cs.CLRecentJun 1, 2026

Benchmarking LLM-as-a-Judge for Long-Form Output Evaluation

Junjie Chen, Yuxi Dong, Haitao Li, Weihang Su +4 more

The paper introduces LongJudgeBench, a new benchmark designed to evaluate the reliability of LLM judges specifically for complex, long-form output evaluation, revealing significant instability gaps in…

View →
cs.CLcs.AIRecentMay 27, 2026

Models That Know How Evaluations Are Designed Score Safer

Katharina Deckenbach, Haritz Puerto, Jonas Geiping, Sahar Abdelnabi

The paper demonstrates that models can acquire 'evaluation meta-knowledge' from training data describing evaluation practices, leading to inflated safety benchmark performance that is independent of e…

View →
cs.CLRecentMay 29, 2026

Preference-Aware Rubric Learning for Personalized Evaluation

Yilun Qiu, Xiaoyan Zhao, Yang Zhang, Yuxin Chen +6 more

The paper introduces PARL, a framework that learns personalized evaluation rubrics directly from raw user interaction histories to accurately assess how well LLM outputs align with subjective, user-sp…

View →
cs.CLcs.AIEmpiricalRecentJun 16, 2026

RubricsTree: Scalable and Evolving Open-Ended Evaluation of Personal Health Agents across Health Memory and Medical Skills

Weizhi Zhang, Zechen Li, Hamid Palangi, Ben Graef +15 more

The authors introduce RubricsTree, a scalable evaluation framework for personal health agents that uses a hierarchical taxonomy of clinically-verifiable rubrics to improve the evaluation of large-scal…

View →
cs.AIRecentMay 27, 2026

Benchmarking AI for low-resource contexts: Thinking beyond leaderboards

Aakash Pant, Kavya Shah, Apoorv Agnihotri, Sneha Nikam +2 more

The paper critiques current AI benchmarking practices for low-resource settings, arguing that evaluation must shift focus from isolated model performance to the holistic performance of the deployed sy…

View →
cs.MAEmpiricalRecentJun 15, 2026

Human-on-the-Bridge: Scalable Evaluation for AI Agents

Fouad Bousetouane

This paper introduces Human-on-the-Bridge (HOB), a scalable evaluation paradigm for agentic AI that amplifies evaluation quality without requiring equally large evaluator models.

View →
cs.CLRecentMay 28, 2026

Generating and Refining Dynamic Evaluation Rubrics for LLM-as-a-Judge

Zijie Wang, Eduardo Blanco

The paper introduces a novel, training-free method to automatically generate fine-grained evaluation rubrics for LLM-as-a-Judge, and further proposes an iterative fine-tuning strategy that significant…

View →
cs.CLcs.AIEmpiricalRecentAug 10, 2026

From Values to Benchmarks: Evaluating Large Language Models for Governmental Use in Dutch

Laurens Samson, Iva Gornishka, Gossa Lô, Yuki M. Asano +1 more

The paper introduces the 'Grip on LLMs' framework, a systematic evaluation suite for Dutch governmental use of large language models, identifying six evaluation dimensions and operationalising them in…

View →
cs.CLcs.AIcs.IREmpiricalRecentOct 15, 2026

AuditKIT: A Unified Evaluation Spine for Benchmarking, Judging, and Comparing Large Language Models and Agentic Systems

Shreeyans Arora, Utsav Avaiya, Zera Lyngkhoi, Hem Gosalia +3 more

The paper presents AuditKIT, a library for evaluating language models and agents by recording various measurements in one result representation.

View →
cs.CLcs.LGEmpiricalRecentJul 30, 2026

(Towards) Scalable Reliable Automated Evaluation with Large Language Models

Bertil Braun, Martin Forell

This paper introduces a novel evaluation framework for Large Language Models using pairwise comparisons and Elo rating system.

View →
stat.MLcs.LGstat.COEmpiricalRecentJun 17, 2026

Quantifying and Auditing LLM Evaluation via Positive--Unlabeled Learning

Zilong Zhang, Yi-Ting Hung, Lei Ding, Chi-Kuang Yeh

The paper proposes a method to correct biased language models by aligning human-verified positives with unlabelled outputs in a fixed embedding space.

View →
cs.AIRecentMay 27, 2026

Better Accuracies, Worse Reasoning: A Step-Level Audit of Medical Chain-of-Thought Distillation

Zhaoyang Jiang, Xuanqi Peng, Fei Teng, Zhizhong Fu +4 more

The paper demonstrates that while distilling large language models for medical QA can significantly improve final answer accuracy, this gain often comes at the cost of factual accuracy and detailed re…

View →
cs.CLcs.CVRecentMay 29, 2026

A Visually Impaired Assistance Benchmark for VLM-as-a-Judge Evaluation

Yi Zhao, Siqi Wang, Zhe Hu, Yushi Li +1 more

The paper introduces VIABLE, the first benchmark for evaluating Vision-Language Models (VLMs) as judges for Visually Impaired Assistance (VIA), finding that current models are largely unreliable and p…

View →