LLM-Based Invariant Testing for Software Functional Bugs
LISA is a novel LLM-based invariant testing framework for software functional bugs, achieving higher bug-detection rates and competitive code coverage than fuzzing and prior LLM-based test generation approaches.
LISA uses machine learning and natural language processing to generate API sequences and program invariants, which is novel compared to prior test generation approaches.
Before reading this…
Applications
- →Software testing
- →Automated bug detection
To understand this paper, make sure you know these concepts first:
- Understanding of software testingfind papers →
- Basic knowledge of machine learningfind papers →
Abstract
More Like ThisManually writing unit tests to uncover functional bugs in software libraries is not only time-consuming but also requires a deep understanding of the intended semantics of the APIs. Heuristic-based test generation methods suffer from low usability because they cannot reason about program semantics or interpret source code and documentation as humans do. Traditional fuzzing techniques like OSS-Fuzz often rely on crashes to detect bugs, but functional bugs do not always cause crashes. To overcome these limitations, we present LISA, a novel LLM-based invariant testing framework for software functional bugs. LISA iteratively generates API sequences and program invariants guided by API n-gram feedback, achieving higher bug-detection rates and competitive code coverage compared with both fuzzing and prior LLM-based test generation approaches, and reporting each finding as a high-confidence bug candidate for developer confirmation.