LISA is a novel LLM-based invariant testing framework for software functional bugs, achieving higher bug-detection rates and competitive code coverage than fuzzing and prior LLM-based test generation…

This study conducts a large-scale longitudinal analysis of CodeQL, finding that while the tool is effective at detecting vulnerabilities, its detection capabilities are not guaranteed to be stable acr…

QASecClaw, a multi-agent LLM system, significantly improves the accuracy of Static Application Security Testing (SAST) by using specialized LLM agents to filter out false positives, achieving an F1 sc…

The paper introduces ARMeta, an LLM-based multi-agent system that automates metamorphic testing for REST APIs documented with OpenAPI, demonstrating its ability to find complementary bugs compared to…

The paper introduces PoVSmith, an agent-based system that uses large language models and call path analysis to automatically generate and assess proof-of-vulnerability tests, significantly improving t…

This paper evaluates the improvement of statistical fault localization by augmenting it with execution features.

The paper introduces SafeAudit, a meta-audit framework that systematically enumerates test cases and uses a quantitative metric to uncover significant residual unsafe behaviors in LLM agents that exis…

The paper introduces TRAILS~, a novel method that improves code correctness validation by grounding LLM reasoning in concrete (input, output) pairs derived from specifications, achieving state-of-the-…

This paper presents NL2Test, a tool that generates executable API regression tests from natural-language scenario descriptions and traffic captures, achieving an 82.4% exact-match rate in industrial s…

This paper characterizes oracle signals in test files of agent-authored pull requests and assesses their impact on merge outcomes.

This paper proposes a context-aware AI framework for generating telecom test scripts that adapt to fine-grained changes in the system using a delta engine and a knowledge graph.

This paper proposes a framework to transform historical real-bug mechanisms into executable feedback targets for LLM-based unit test generation, improving real-bug detection over various baselines.

This paper systematically surveys adaptive and AI-augmented security testing, concluding that a major gap exists—structural-adaptive fragmentation—where current systems fail to integrate structural pr…

The paper introduces Code-QA-Bench, a novel framework that rigorously separates genuine code reasoning from mere documentation memorization in repository-level code understanding benchmarks.

This paper proposes CoHarden, a co-generation framework for automated program repair that uses a lax signal as an in-loop convergence criterion to prevent lax regressions.

The paper introduces Test-Driven Forensics, an approach that treats forensic expectations as executable tests to detect and measure the degradation of repeatability and confidence in digital forensic…

This paper explores the causes of inconsistent vulnerability scanner findings in the open-source ecosystem.

The paper introduces FVSpec, a large-scale benchmark that translates thousands of real-world Python property-based tests into formal Lean 4 specifications to evaluate AI models for formal software ver…