20 results for “Task exchangeability”
CS papers onlyHybrid search: Keyword + semantic, ranked by combined score.ⓘ
Want pure semantic search? Try claim verification →
This paper proposes statistical principles for using synthetic data in scientific research with provable validity guarantees.
Tomer Keren, Nitay Calderon, Asaf Yehudai, Yotam Perlitz +2 more
The paper introduces TASTE, an automatic task synthesis method that generates challenging agent benchmarks by evolving tool sequences, demonstrating that existing benchmarks are saturated and that TAS…
Kou Shi, Ziao Zhang, Shiting Huang, Avery Nie +6 more
The paper introduces AsyncTool, a new benchmark designed to evaluate LLM agents' ability to handle multiple, concurrent tasks with delayed tool feedback, demonstrating that asynchronous coordination i…
Likai Wang, Dezhao Su, Jing Hu, Yunyun Wan +2 more
A framework for measuring human contributions in human-agent joint production is proposed.
Weihuang Zheng, Tianyuan Zou, Eileen Ye, Alphet Liu +4 more
The paper introduces E-Bench, a synthetic benchmark for evaluating multi-step tool use in Large Language Models across three product domains.
Sadra Sabouri, Hosein Hasanbeig, Adam Sage, Shreya Sinha +5 more
The paper presents a method to compile agent runs into reusable neuro-symbolic programs (NSPs) to improve efficiency and consistency.
The paper introduces Croissant Tasks, a declarative metadata format designed to achieve conceptual reproducibility in machine learning by abstracting problem specifications from brittle implementation…
This paper compares ten deterministic and two LLM-based methods for estimating robot suitability for tasks.
ToolBench-X is a benchmark for evaluating agents under recoverable reliability hazards
This paper studies the complexity of Value-based Task Selection (VTS) in real-time systems.
This paper introduces the Goal-Oriented Dialogue Runtime (GODR), a framework-neutral design pattern for managing complex, multi-domain, interruptible conversations with multiple interdependent objecti…
Sina Mavali, David Pape, Jonathan Evertz, Samira Abedini +4 more
The paper introduces the Task Alignment Benchmark (TAB) to evaluate terminal agents' ability to selectively follow relevant environmental instructions while ignoring misleading distractors, revealing…
Mengyu Zheng, Kai Han, Boxun Li, Haiyang Xu +12 more
The paper introduces Claw-SWE-Bench, a benchmark and adapter protocol for evaluating heterogeneous agent harnesses in software engineering tasks.
The paper introduces Clover, a code completion tool that logs students' interactions and offers attention checks to promote reflective engagement during programming tasks.
Nhat Minh Le, Yisen Xu, Zhijie Wang, Tse-Hsun +1 more
This paper evaluates the performance of large language models on popular benchmarks and finds that only a small percentage of the performant implementations are significantly faster than canonical sol…
The paper introduces Spine, an interface and runtime that improves the efficiency of long-horizon coding tasks by aligning model context with subtask nesting and completion.