ArXivCSExplorer
☆☆Bookmarks🏆RSSHow to UseFAQ
Built with and by Teycir Ben Soltane•
How to Use•FAQ•GitHub•arXiv.org•
Share:

20 results for “benchmark suite”

CS papers only

Hybrid search: Keyword + semantic, ranked by combined score.ⓘ

Want pure semantic search? Try claim verification →

cs.SEEmpiricalRecentJul 8, 2026

Rethinking Code Performance Benchmarks for LLMs

Nhat Minh Le, Yisen Xu, Zhijie Wang, Tse-Hsun +1 more

This paper evaluates the performance of large language models on popular benchmarks and finds that only a small percentage of the performant implementations are significantly faster than canonical sol…

View →
cs.SEEmpiricalRecentOct 27, 2026

BenchMeUp: Building Software Engineering Benchmarks from Natural-Language Specifications

Islem Bouzenia, Michael Pradel

BenchMeUp is a system that automatically creates software-engineering benchmarks from natural-language specifications.

View →
cs.LGcs.AIEmpiricalRecentNov 5, 2026

XStack: A Benchmark for ML Repo Migration

Linrong Cai, Taiming Lu, Haozhe Chen, Zhuang Liu

The paper introduces XStack, a benchmark that evaluates agents on migrating ML repositories between PyTorch and JAX ecosystems.

View →
cs.AIcs.CLcs.HCEmpiricalRecentJun 21, 2026

MacAgentBench: Benchmarking AI Agents on Real-World macOS Desktop

Yikun Fu, Bowen Fu, Zhenyu Wu, Shuang Cheng +9 more

The paper presents MacAgentBench, a comprehensive macOS agent benchmark for evaluating computer use agents.

View →
cs.AIEmpiricalRecentJul 26, 2026

E-Bench: Benchmarking Multi-Step Tool-Use Agents in Real-World Product Scenarios

Weihuang Zheng, Tianyuan Zou, Eileen Ye, Alphet Liu +4 more

The paper introduces E-Bench, a synthetic benchmark for evaluating multi-step tool use in Large Language Models across three product domains.

View →
cs.SEcs.AIRecentMay 28, 2026

CodeGolf Bench: A Multi-Language Benchmark for Evaluating Concise Code Generation Capabilities of Large Language Models

Vedant Padwal

The paper introduces CodeGolf Bench, a novel multi-language benchmark using code golf to measure LLMs' ability to generate highly concise and efficient code, showing that reasoning models significantl…

View →
cs.DBcs.AIcs.CLEmpiricalRecentJul 24, 2026

DBA-Bench: A Production-Fidelity Benchmark for LLM-Based Database Operations Agents

Junming Chen, Junyang Jiang, Xu Chen, Zibo Liang +1 more

The paper introduces DBA-Bench, a benchmark for evaluating database agents with production fidelity, outcome-first evaluation, and controlled scenario reproducibility.

View →
cs.AIRecentMay 27, 2026

Benchmarking AI for low-resource contexts: Thinking beyond leaderboards

Aakash Pant, Kavya Shah, Apoorv Agnihotri, Sneha Nikam +2 more

The paper critiques current AI benchmarking practices for low-resource settings, arguing that evaluation must shift focus from isolated model performance to the holistic performance of the deployed sy…

View →
cs.CLcs.SEEmpiricalRecentJun 22, 2026

EnterpriseClawBench: Benchmarking Agents from Real Workplace Sessions

Jincheng Zhong, Weizhi Wang, Che Jiang, Kai Tian +4 more

The paper introduces EnterpriseClawBench, an enterprise agent benchmark with 852 tasks and evaluation protocol, achieving a best configuration score of 0.663.

View →
cs.SEcs.AIRecentMay 28, 2026

Code-QA-Bench: Separating Code Reasoning from Documentation Memorization in Repository-Level QA

Jun Zhang, JianYing Qu, Hanwen Du, Zhongkai Sun +2 more

The paper introduces Code-QA-Bench, a novel framework that rigorously separates genuine code reasoning from mere documentation memorization in repository-level code understanding benchmarks.

View →
cs.AIRecentMay 28, 2026

FinVerBench: Benchmark Validity and Calibration in Large Language Model Financial Statement Verification

Silu Panda

The paper introduces FinVerBench, a comprehensive benchmark for financial statement verification, concluding that successful verification requires calibrated judgment under realistic observational con…

View →
cs.SEEmpiricalRecentJun 15, 2026

No Resource, No Benchmarks, No Problem? Evaluating and Improving LLMs for Code Generation in No-Resource Languages

Alessandro Giagnorio, Alberto Martin-Lopez, Gabriele Bavota

This paper investigates code generation for no-resource programming languages using large language models.

View →
cs.LGcs.CLEmpiricalRecentJun 10, 2026

Claw-SWE-Bench: A Benchmark for Evaluating OpenClaw-style Agent Harnesses on Coding Tasks

Mengyu Zheng, Kai Han, Boxun Li, Haiyang Xu +12 more

The paper introduces Claw-SWE-Bench, a benchmark and adapter protocol for evaluating heterogeneous agent harnesses in software engineering tasks.

View →
cs.SEcs.AIcs.CLRecentMay 31, 2026

BenchEvolver: Frontier Task Synthesis via Solution-Centric Evolution

Yangzhen Wu, Aaron J. Li, Wenjie Ma, Li Cao +9 more

BenchEvolver introduces a solution-centric evolutionary framework to automatically transform saturated coding benchmarks into significantly harder, high-quality, and diverse evaluation suites.

View →