BrickBench: Evaluating Agentic Brick Design
A benchmark for evaluating text-conditioned LEGO-set design agents that can produce physically buildable designs.
This paper proposes a novel benchmark for evaluating text-conditioned LEGO-set design agents, which is a new area of research.
Before reading this…
Applications
- →Improving text-conditioned design agents for LEGO-set design
To understand this paper, make sure you know these concepts first:
- Understanding of text-conditioned designfind papers →
- Knowledge of LEGO-set designfind papers →
Abstract
More Like ThisWe propose BrickBench, a benchmark for agentic text-conditioned LEGO-set design. Given a prompt, an agent is tasked with producing an assembly that not only satisfies semantic and design criteria, but that can also be physically built. To do so, it must select parts from a discrete library and reason jointly about local and global constraints. We score validity, alignment, and design across three settings that vary in scale and part availability. We provide BrickAgent, an environment for coding agents to construct, inspect, and validate their designs. We find that leading agents largely satisfy verifiable physical and semantic requirements, but fall short of human designs. We release our benchmark and environment at http://www.brickben.ch