OpenDraco: A Platform for Engineering and Experimenting with Configurable Multi-Agent Architectures for Software Engineering
OpenDraco is an open-source platform that integrates activities for experimenting with LLM-based multi-agent systems.
OpenDraco provides a novel platform for integrating activities for experimenting with LLM-based multi-agent systems, addressing the limited end-to-end support in existing approaches.
Before reading this…
Applications
- →Automated program repair (APR)
- →Sequential baseline and reconstructed configurations
To understand this paper, make sure you know these concepts first:
- Knowledge of LLM-based multi-agent systemsfind papers →
- Understanding of the experimentation lifecyclefind papers →
Abstract
More Like ThisExperimenting with LLM-based multi-agent systems (MAS) involves an iterative cycle of modeling configurations, executing and evaluating them, comparing results, and refining system designs. Existing approaches provide limited end-to-end support for this experimentation lifecycle. We present OpenDraco, an open-source platform that integrates these activities into a single workflow. OpenDraco represents agents, prompts, tools, models, parameters, and communication topology in exportable, versioned JSON specifications, translated into executable LangGraph workflows supporting different interaction structures. Users can modify and re-evaluate configurations using local or cloud-based LLMs. We instantiate OpenDraco for automated program repair (APR) and evaluate a sequential baseline and seven reconstructed configurations on synthetic and real-world SWE-bench Lite issues. The results reveal substantial cost differences among configurations with similar repair outcomes, while the baseline resolves 12 of 77 SWE-bench issues using open-source LLMs. Additional proof-of-concept tasks illustrate applicability beyond APR. OpenDraco supports iterative experimentation with heterogeneous MAS configurations, enabling researchers to examine their outcomes and costs within a common environment. Demonstration video: https://youtu.be/278sOR4lBEk.