The paper introduces CodeGolf Bench, a novel multi-language benchmark using code golf to measure LLMs' ability to generate highly concise and efficient code, showing that reasoning models significantl…

This paper evaluates the performance of large language models on popular benchmarks and finds that only a small percentage of the performant implementations are significantly faster than canonical sol…

This paper reports on SciGMark 1.5, a benchmark study of specialized and generic implementations in modern languages, examining the consequences of various generic-realization strategies and extending…

The paper introduces Code-QA-Bench, a novel framework that rigorously separates genuine code reasoning from mere documentation memorization in repository-level code understanding benchmarks.

The paper proposes a generic memory framework for advanced static program analysis, supporting various memory behaviors from different programming languages using Abstract Interpretation theory.

KoAT is a tool that automatically infers complexity bounds and proves termination of integer programs using an alternating modular analysis approach and a portfolio of techniques.

The paper proposes Fuzzy-Function Programming and introduces Program-as-Weights (PAW), a compact, locally-executable neural artifact for everyday programming tasks.

The paper introduces FVSpec, a large-scale benchmark that translates thousands of real-world Python property-based tests into formal Lean 4 specifications to evaluate AI models for formal software ver…

The paper introduces REBench, a comprehensive, standardized benchmark dataset designed to enable fair and rigorous evaluation of Large Language Models (LLMs) on complex binary reverse engineering task…

The paper introduces RealVuln, a benchmark that demonstrates a clear three-tier performance hierarchy for security scanners on real-world code, with specialized tools significantly outperforming gener…

This paper analyzes various source-to-bytecode obfuscation techniques for Erlang, demonstrating that effective protection relies on exploiting the representational gaps between high-level semantics an…

The paper proposes TRAVEL, a framework for automated C-to-Rust translation using Monte Carlo Tree Search and reinforcement learning, improving accuracy and success rate.

The paper introduces Spaghetti Architect, a tool that generates controlled code datasets for machine learning models by deliberately adding redundancy, messiness, and difficulty.

The paper introduces SEC-bench Pro, a rigorous benchmark for evaluating LLM-based bug hunting on complex software, finding that even advanced agents struggle with long-horizon security tasks.

The largest reported verification campaign for the Rust standard library is presented, integrating complementary tools and analyzing effectiveness.

Elevator is a novel, deterministic binary translator that statically translates entire x86-64 executables to AArch64 by considering all possible interpretations of every byte, eliminating the need for…