A framework called Restor is presented, which generates executable test assertions from a single observed request-response pair in a black-box REST API setting using a fine-tuned Large Language Model.

The paper introduces ARMeta, an LLM-based multi-agent system that automates metamorphic testing for REST APIs documented with OpenAPI, demonstrating its ability to find complementary bugs compared to…

The paper enhances REST API fuzzing by introducing novel automated oracles that detect access policy violations and execute traditional injection attacks, successfully identifying security flaws in mu…

This paper presents NL2Test, a tool that generates executable API regression tests from natural-language scenario descriptions and traffic captures, achieving an 82.4% exact-match rate in industrial s…

This paper evaluates the performance of large language models on popular benchmarks and finds that only a small percentage of the performant implementations are significantly faster than canonical sol…

LISA is a novel LLM-based invariant testing framework for software functional bugs, achieving higher bug-detection rates and competitive code coverage than fuzzing and prior LLM-based test generation…

This paper proposes using large language models (LLMs) to generate and compositionally verify software implementations directly from natural language specifications, showing promising preliminary resu…

This paper characterizes oracle signals in test files of agent-authored pull requests and assesses their impact on merge outcomes.

The paper introduces Map Reduce Graph (MRG), an unsupervised method that automatically models and secures HTTP REST APIs by learning their structure from real-world traffic, achieving high accuracy an…

The paper introduces Sieve, a system that uses a large language model (LLM) to generate executable query code from natural language security questions, significantly improving the ability to perform c…

This paper proposes a framework to transform historical real-bug mechanisms into executable feedback targets for LLM-based unit test generation, improving real-bug detection over various baselines.

The paper introduces TRAILS~, a novel method that improves code correctness validation by grounding LLM reasoning in concrete (input, output) pairs derived from specifications, achieving state-of-the-…

The paper introduces SafeAudit, a meta-audit framework that systematically enumerates test cases and uses a quantitative metric to uncover significant residual unsafe behaviors in LLM agents that exis…

This paper formalizes token optimization as a multi-objective constrained transformation problem for LLM-based Oracle-to-PostgreSQL migration, demonstrating that adaptive routing offers the best balan…

The paper introduces POLARIS, a novel framework that systematically generates comprehensive and verifiable safety tests for LLMs by formalizing natural language policies into First-Order Logic and exp…

The paper introduces CodeGolf Bench, a novel multi-language benchmark using code golf to measure LLMs' ability to generate highly concise and efficient code, showing that reasoning models significantl…

The paper introduces an automated framework demonstrating that LLM system instructions are vulnerable to encoding attacks, where structured output requests can bypass safety refusals and leak sensitiv…

The paper addresses the reliability of open-weight LLMs for power system code generation by identifying structured API-knowledge boundary errors and proposing a boundary-aware intervention that signif…

The paper presents Tahoe, a system that optimizes Text-to-SQL performance through dynamic data management and hint learning.