ArXivCSExplorer
☆☆Bookmarks🏆RSSHow to UseFAQ
Built with and by Teycir Ben Soltane•
How to Use•FAQ•GitHub•arXiv.org•
Share:

20 results for “evaluation pipeline”

CS papers only

Hybrid search: Keyword + semantic, ranked by combined score.ⓘ

Want pure semantic search? Try claim verification →

cs.IREmpiricalRecentJul 1, 2026

Trie-based Experiment Plans for Efficient IR Pipeline Experiments

Irene Anu, Craig Macdonald

This paper describes the use of a trie data structure to enhance experiment efficiency in comparative pipeline experiments for cascading retrieval pipelines using PyTerrier, observing a 26% reduction…

View →
cs.HCcs.AIEmpiricalRecentJun 17, 2026

A Clinician-Centered Pipeline for Annotation and Evaluation in Ultrasound AI Studies

Fangyijie Wang, Jianjun Yu, Wentao Shi, Haixia Huang +3 more

A clinician-centered pipeline is proposed for remote annotation and evaluation of ultrasound AI studies.

View →
cs.AIRecentJun 1, 2026

BADGER: Bridging Agentic and Deterministic Evaluation for Generative Enterprise Reasoning

Shannon Serrao, Soumitra Chatterjee, Dorina Strori, Abhishek Sharma +1 more

BADGER is a unified, production-grade evaluation framework that integrates text-to-SQL assessment with agentic behavior evaluation, significantly outperforming existing benchmarks on industry queries.

View →
cs.CLcs.AIRecentMay 28, 2026

MedCase-Structured: A Text-to-FHIR Dataset for Benchmarking Diagnostic Reasoning in Clinically Realistic EHR Settings

Valentina Bui Muti, Eugénie Dulout, Ziquan Fu

The paper introduces MedCase-Structured, a synthetic, FHIR-formatted dataset designed to benchmark diagnostic reasoning in realistic EHR settings, showing that LLMs perform worse on structured data th…

View →
cs.CLcs.IREmpiricalRecentJul 15, 2026

DS@GT ARC at LongEval: Citation Integrity and Factual Grounding in Scientific QA

Brandon Michaels, Brendon Johnson

DS@GT ARC evaluates the effectiveness of Corrective RAG (CRAG) and CiteFix in improving citation faithfulness and answer grounding in Retrieval-Augmented Generation (RAG) QA systems, revealing a trade…

View →
cs.CVcs.LGeess.IVRecentJun 3, 2026

An Open-Source Two-Stage Computer Vision Pipeline for Fine-Grained Vehicle Classification using Vision Transformers

Gandhimathi Padmanaban, Fred Feng

This paper presents an open-source computer vision pipeline for classifying vehicle body types from naturalistic roadway video.

View →
cs.AIRecentJun 1, 2026

Consistency evaluation of benchmarks used for causal discovery

Yuzhe Zhang, Chihui Chen, Lina Yao, Chen Wang

This paper systematically evaluates the consistency of popular causal discovery benchmarks against real-world scientific literature, revealing significant variability in their accuracy.

View →
cs.ARcs.AIcs.LGEmpiricalRecentJun 11, 2026

VHDLSuite: Unified Pipeline for LLM VHDL Generation with Data Synthesis and Evaluation

Yijun Shen, Minghao Shao, Yichen Zhao, Zhuoyan Yu +3 more

The paper introduces VHDLSuite, an infrastructure for evaluating Large Language Models in VHDL, including a data pipeline, benchmark, and evaluation framework.

View →
cs.AIRecentMay 31, 2026

"Skill issues'': data-centric optimization of lakehouse agents

Nicole Rose Schneider, Davide Ghilardi, Giacomo Piccinini, Jacopo Tagliabue

The paper introduces a data-centric optimization pipeline to improve coding agents' ability to interact with a branching lakehouse, showing significant accuracy gains by treating agent evaluation as a…

View →
cs.AIcs.IRRecentMay 28, 2026

Rethinking Literature Search Evaluation: Deep Research Helps, and Human Citation Lists Are Not a Ground Truth

Gaurav Sahu, Laurent Charlin, Christopher Pal

The paper introduces a Deep Research pipeline that significantly improves literature search recall and demonstrates that human-curated citation lists are often unreliable and do not serve as a true gr…

View →
cs.CRcs.AIcs.LGMar 29, 2026

Evaluating Prompt Injection Defenses for Educational LLM Tutors: Security-Usability-Latency Trade-offs

Alexandre Cristovão Maiorano

The paper evaluates prompt-injection defenses for educational LLM tutors, demonstrating that optimal security requires balancing adversarial robustness, usability, and latency, and proposing a compreh…

View →
cs.CRcs.LGcs.SEApr 21, 2026

Evaluating LLM-Generated Obfuscated XSS Payloads for Machine Learning-Based Detection

Divyesh Gabbireddy, Suman Saha

This paper proposes a structured pipeline using LLMs to generate and evaluate obfuscated XSS payloads, demonstrating that while LLMs can generate samples, they currently struggle to ensure payloads ma…

View →
cs.AIcs.CLRecentMay 28, 2026

GTA: Generating Long-Horizon Tasks for Web Agents at Scale

Tenghao Huang, Kung-Hsiang Huang, Prafulla Kumar Choubey, Yilun Zhou +3 more

The paper introduces GTA, a scalable framework for generating realistic, multi-hop web-agent tasks with dense, executable trajectories, addressing the current lack of process-level supervision in web…

View →
cs.SEEmpiricalRecentJul 2, 2026

Epic-Organized vs. Requirement-Aligned Gherkin: An Empirical Evaluation of LLM-Based Acceptance Criteria Generation

Shahbaz Siddeeq, Mateen Abbasi, Jussi Rasku, Zheying Zhang +3 more

This paper compares the quality and coverage of epic-organized LLM-generated Gherkin acceptance criteria with requirement-aligned generation, using four requirements documents from the PURE dataset.

View →
cs.IRcs.AIEmpiricalRecentJul 28, 2026

MARS: Multi-Agent Re-ranking for Repeat-Order Food Delivery Recommendation

Jiahao Tian, Zhenkai Wang

This paper introduces MARS, a framework for repeat-order food delivery recommendation that combines pre-trained language models with collaborative retrieval and contextual filtering.

View →
eess.ASEmpiricalRecentJun 12, 2026

Who Spoke When in Multi-Conversation: Target Speaker Tagging Task and Benchmark

Minjae Lee, Hee-Soo Heo, Youngki Kwon, Han-Gyu Kim +2 more

The paper introduces Target Speaker Tagging (TST), a task that combines speaker diarization, verification, and identification into a single workflow for multi-speaker conversations. It presents TST-Be…

View →
cs.CLcs.AIRecentJun 1, 2026

Who Annotates in NLP? A Large-scale Assessment of Human Annotation Reporting between 2018 and 2025

Maria Kunilovskaya, Gagan Bhatia, Lisa Sophie Albertelli, Yanran Chen +9 more

This paper conducts a large-scale audit of human annotation reporting in NLP, finding that while reporting has improved, critical details needed to assess annotation validity, such as training and agr…

View →
cs.SDEmpiricalRecentJul 8, 2026

MMGenre: Benchmarking Singing Voice Synthesis across Multiple Musical Genres

Wenhao Feng, Yuxun Tang, Jiatong Shi, Qin Jin

The paper introduces MMGenre, a benchmark for multi-genre singing voice synthesis diagnosis, revealing limited genre discrimination and proposing lightweight genre-specific continued training.

View →
cs.CLcs.AIcs.CREmpiricalRecentOct 1, 2026

KaliBench: A Fine-Grained Benchmark for Cybersecurity Tool Use on Kali Linux with Runtime-Free Verifiable Rewards

Pengfei Li, Naufal Suryanto, Sicheng Zhang, Muzammal Naseer

This paper introduces KaliBench, a fine-grained benchmark and dataset for evaluating natural-language-to-CLI translation in cybersecurity workflows.

View →
cs.ROEmpiricalRecentJun 22, 2026

LIBERO-Safety: A Comprehensive Benchmark for Physical and Semantic Safety in Vision-Language-Action Models

Rongxu Cui, Zongzheng Zhang, Jingrui Pang, Haohan Chi +10 more

This paper introduces a safety benchmark and dataset to evaluate the safety of Vision-Language-Action models.

View →