BUSTA: Teaching Small Language Models to Reason About Binary Execution
This paper explores the use of large language models for reverse engineering tasks and presents a targeted training approach to improve their performance.
This paper presents a novel approach to training large language models for reverse engineering tasks, which is distinct from their typical use cases.
Before reading this…
Applications
- →Reverse engineering
To understand this paper, make sure you know these concepts first:
- Knowledge of large language models and reverse engineeringfind papers →
Abstract
More Like ThisLarge Language Models (LLM) are increasingly applied to Reverse Engineering (RE) workflows. RE, however, differs from common LLM coding tasks as it requires models to reason about low-level program behavior. Furthermore, the deployment of tooling in resource-constrained sandboxed RE environments limits the ability to use frontier-scale solutions. We examine whether a targeted LLM training approach can transfer distinct RE skills to open models via inverse basic-block execution: given an x86-64 basic block and a desired output register state, the model predicts the input register values that produce it. Fine-tuning \texttt{Qwen3-1.7B} and \texttt{Nemotron-3-Nano} with Group Relative Policy Optimization (GRPO) raises knowable-register prediction accuracy from 73.2\% to 85.3\% and from 63.3\% to 95.2\%, respectively. Frontier models still score highest (up to 99.2\%), but our fine-tuned \texttt{Nemotron-3-Nano} comes within four points at a fraction of the parameter count. We release both fine-tuned models and the training dataset.