The paper introduces ARMeta, an LLM-based multi-agent system that automates metamorphic testing for REST APIs documented with OpenAPI, demonstrating its ability to find complementary bugs compared to…

LISA is a novel LLM-based invariant testing framework for software functional bugs, achieving higher bug-detection rates and competitive code coverage than fuzzing and prior LLM-based test generation…

This paper proposes a context-aware AI framework for generating telecom test scripts that adapt to fine-grained changes in the system using a delta engine and a knowledge graph.

This paper systematically surveys adaptive and AI-augmented security testing, concluding that a major gap exists—structural-adaptive fragmentation—where current systems fail to integrate structural pr…

This paper proposes an approach to automate the development of app-specific voice assistants using GUI test code and large language models.

The paper introduces POLARIS, a novel framework that systematically generates comprehensive and verifiable safety tests for LLMs by formalizing natural language policies into First-Order Logic and exp…

This study conducts a large-scale longitudinal analysis of CodeQL, finding that while the tool is effective at detecting vulnerabilities, its detection capabilities are not guaranteed to be stable acr…

This paper presents NL2Test, a tool that generates executable API regression tests from natural-language scenario descriptions and traffic captures, achieving an 82.4% exact-match rate in industrial s…

QASecClaw, a multi-agent LLM system, significantly improves the accuracy of Static Application Security Testing (SAST) by using specialized LLM agents to filter out false positives, achieving an F1 sc…

This paper proposes a framework to transform historical real-bug mechanisms into executable feedback targets for LLM-based unit test generation, improving real-bug detection over various baselines.

The paper introduces FVSpec, a large-scale benchmark that translates thousands of real-world Python property-based tests into formal Lean 4 specifications to evaluate AI models for formal software ver…

This paper proposes CoHarden, a co-generation framework for automated program repair that uses a lax signal as an in-loop convergence criterion to prevent lax regressions.

The paper introduces SafeAudit, a meta-audit framework that systematically enumerates test cases and uses a quantitative metric to uncover significant residual unsafe behaviors in LLM agents that exis…

This paper proposes MineValiCoder, a collaborative closed-loop TDD framework using mutual reinforcement of test-case quality and code quality to address stochasticity in Large Language Model-based Tes…

This paper characterizes oracle signals in test files of agent-authored pull requests and assesses their impact on merge outcomes.

The paper introduces GUITestScape, a comprehensive benchmark for exploratory GUI testing, and GUIJudge, an open-set evaluator that significantly improves the assessment of AI agents' defect detection…

The paper introduces PoVSmith, an agent-based system that uses large language models and call path analysis to automatically generate and assess proof-of-vulnerability tests, significantly improving t…

The paper introduces Test-Driven Forensics, an approach that treats forensic expectations as executable tests to detect and measure the degradation of repeatability and confidence in digital forensic…