N-Version Programming with Coding Agents
This paper revisits N-version programming with AI coding agents and finds substantial common-mode failures but also practical benefits.
Provides empirical evidence for the usefulness of N-Version Programming with coding agents
Before reading this…
Applications
- →Software engineering, AI coding agents
To understand this paper, make sure you know these concepts first:
- Understanding of N-version programming, AI coding agentsfind papers →
Abstract
More Like ThisThis paper revisits the classical concept on N-version programming in the setting of contemporary AI coding agents. Revisiting the seminal Knight-Leveson experiment, we study whether diversity across agent systems, models, and implementation languages creates diverse failure modes. Using the Knight-Leveson's, Launch Interceptor Program Specification, we evaluate 48 agent-generated implementations on a shared oracle and a campaign of 1,000,000 randomized test inputs. The results show substantial common-mode failure, along the findings of Knight-Leveson. Further analysis that many of those co-occuring failures can be traced to where is specification is particularly hard or ambiguous. We also demonstrate that diversity from coding agents provides practical benefit: across majority voting three-version units, the mean failure count drops from 387.44 for single versions to 130.99 for triples, and 11,844 N-version units exhibit zero observed failures. Our original results is the strongest evidence to date that N-Version Programming with coding agents is a useful engineering strategy.