This paper presents an empirical study on how developers respond to agentic code reviews using CodeRabbit, revealing mixed reception and opportunities for improvement.

The paper introduces Code-QA-Bench, a novel framework that rigorously separates genuine code reasoning from mere documentation memorization in repository-level code understanding benchmarks.

This paper proposes conversational AI review assistants for code review, systems that engage in conversation with developers instead of just generating comments.

This paper synthesizes practitioner discourse at scale to build a causal model explaining the impact of AI on code review, recovering mechanisms behind observed trends.

The paper introduces CodeGolf Bench, a novel multi-language benchmark using code golf to measure LLMs' ability to generate highly concise and efficient code, showing that reasoning models significantl…

The paper introduces TRAILS~, a novel method that improves code correctness validation by grounding LLM reasoning in concrete (input, output) pairs derived from specifications, achieving state-of-the-…

The paper evaluates code language model-based detection of vulnerability-fixing commits (VFCs) using a unified benchmark and concludes that code changes alone are insufficient for accurate detection,…

This paper introduces CORE-Bench, a comprehensive benchmark for code retrieval in agentic coding.

LISA is a novel LLM-based invariant testing framework for software functional bugs, achieving higher bug-detection rates and competitive code coverage than fuzzing and prior LLM-based test generation…

The paper organizes the first Poisoned Chalice of LLM Evaluation Competition to detect a large language model's prior exposure to training data without access to the full training corpus.

This study analyzes over 20,000 real-world coding sessions to show that AI coding agents frequently fail users through subtle misalignment, requiring constant manual correction even when major system…

The paper introduces Spaghetti Architect, a tool that generates controlled code datasets for machine learning models by deliberately adding redundancy, messiness, and difficulty.

This paper characterizes oracle signals in test files of agent-authored pull requests and assesses their impact on merge outcomes.

This survey analyzed 132 web application security tutorials, finding that most lack concrete implementation details and recommending that the presence of runnable code and links to official resources…

The paper introduces FVSpec, a large-scale benchmark that translates thousands of real-world Python property-based tests into formal Lean 4 specifications to evaluate AI models for formal software ver…

The paper introduces MOSAIC-Bench, a benchmark demonstrating that coding agents can ship exploitable code by complying with seemingly innocuous, staged tasks, a vulnerability that is not easily mitiga…

The largest reported verification campaign for the Rust standard library is presented, integrating complementary tools and analyzing effectiveness.

This paper presents Forge, a pipeline that uses formal verification to ensure the safety and certification of LLM-generated Java code.

SEMBridge is a tagless-final framework that allows a single executable object program to generate multiple program semantics, including weakest-precondition and bounded-checking interpretations, ensur…