This paper investigates the prevalence and impact of relation chains in software review workflows using data from Gerrit, finding that they increase merge times and review effort propagation.

The paper introduces FORCEBENCH, a new stress test designed to evaluate whether cited sources genuinely warrant the strength of a claim, revealing that standard citation evaluation methods often fail…

This paper empirically evaluates LLM-generated reviews for academic papers, finding that while LLM reviews show some alignment with human ones, authors can effectively 'game' the system using iterativ…

The paper proposes a taxonomy for AI-human collaboration in scientific evaluation and introduces the Paper Assistant Tool (PAT) to accelerate verification and review process in scientific research.

The paper empirically investigates whether AI-generated reviews can improve the drafting process of academic papers, finding that AI reviews cover many human-identified issues but also introduce novel…

This paper synthesizes practitioner discourse at scale to build a causal model explaining the impact of AI on code review, recovering mechanisms behind observed trends.

The paper evaluates an automated legal triage system (FETCH) that uses follow-up questions, demonstrating that while low-cost LLMs are effective for classification, generating high-quality questions r…

This paper presents an empirical study on how developers respond to agentic code reviews using CodeRabbit, revealing mixed reception and opportunities for improvement.

This paper proposes conversational AI review assistants for code review, systems that engage in conversation with developers instead of just generating comments.

The paper introduces FormInv, a measurement protocol that reveals significant semantic inconsistencies in existing mathematical reasoning benchmarks, showing that standard accuracy metrics fail to cap…

The paper introduces PRAIB, a benchmark that demonstrates that LLM-generated peer reviews, while often verbose, systematically diverge from human norms by being less variable, positively biased, and f…

DS@GT ARC evaluates the effectiveness of Corrective RAG (CRAG) and CiteFix in improving citation faithfulness and answer grounding in Retrieval-Augmented Generation (RAG) QA systems, revealing a trade…

The paper introduces an Item Response Theory (IRT)-based indicator that effectively identifies likely mislabeled items in existing LLM benchmarks, revealing systematic errors in labeling and model spe…

The paper introduces HOPM, a hierarchical online prompt mutation framework that significantly improves the performance of language models in high-stakes evidence document generation by integrating dua…

The paper demonstrates that while distilling large language models for medical QA can significantly improve final answer accuracy, this gain often comes at the cost of factual accuracy and detailed re…

This paper introduces a framework to audit source-dependence in multi-source RAG systems, demonstrating that disagreement across institutional sources is a common and critical failure mode that curren…

This paper tests the assumption that evaluation is easier than generation in LLM-as-a-Judge and self-evaluation pipelines using a controlled in-context QA setting and reveals that evaluation attends t…

CRITIC-R1 introduces a structured critic framework that treats RAG critique as an explicit error diagnosis problem using reinforcement learning, significantly improving answer quality over strong RAG…