GitOfThoughts: Version-Controlled Reasoning and Agent Memory You Can Replay, Diff, and Merge
The authors introduce GitOfThoughts, a system that stores an agent's reasoning tree as a git repository, and investigate the role of memory in large language model reasoning.
The authors' use of a git repository to store and replay reasoning trees is a novel approach to understanding large language model reasoning.
Before reading this…
Applications
- →Improving the understanding of large language model reasoning
- →Developing more efficient and effective memory formats for large language models
To understand this paper, make sure you know these concepts first:
- Understanding of large language models and their reasoning processesfind papers →
Abstract
More Like ThisLarge language model (LLM) reasoning is ephemeral: chains of thought vanish with the context window, pruned search branches leave no record, and memory buffers cannot be diffed, merged, or audited. Every other complex software process (code, infrastructure, data, experiments) is version-controlled; reasoning is not. We introduce GitOfThoughts, which stores an agent's reasoning tree as a git repository: every scored thought is a commit, scores are notes, outcomes are tags, and retrieval is "git log" over the agent's own history. This makes reasoning replayable, auditable, and mergeable across agents at near-zero engineering cost. We then ask the harder question: does memory, in any substrate, actually improve accuracy? Across five substrates (none, markdown, vector, graph, git), two benchmarks, two model scales, and pre-registered replications, the answer for novel problems is no. No memory format reliably helps, and a promising early result collapsed under its own pre-registered replication. Memory pays only above what we call the copyability threshold: when the retrieved case is a near-duplicate of the current problem (similarity >~ 0.8), accuracy jumps sharply; below it, nothing. The gain is answer retrieval, not method transfer: a 4.5x larger model doubles the near-duplicate payoff yet still cannot extract a transferable method from a worked example. The only general lever we find is test-time sampling. The case for git-as-substrate is therefore auditability, provenance, and mergeability at accuracy parity. We document a retracted result and a refuted hypothesis to model the evaluation standard we hold ourselves to.