Test-Time Self-Play: Scaling without a Verifier
The paper introduces Test-Time Self-Play, an approach to improve large language model performance without relying on a verifier.
Introduces a new approach to test-time scaling that doesn't rely on a verifier.
Before reading this…
Applications
- →Improving language model performance
- →Test-time scaling
To understand this paper, make sure you know these concepts first:
- Large language modelsfind papers →
- Self-playfind papers →
Abstract
More Like ThisTest-time scaling methods spend additional compute at deployment to improve a large language model's performance on a hard target problem. Most methods rely on a verifier that scores candidate solutions for correctness, using it to select among or refine the model's outputs. In many deployment regimes, however, a strong verifier on the target problem is unavailable. We introduce Test-Time Self-Play (TTSP), an approach to test-time scaling using asymmetric self-play without reliance on a verifier. On a difficulty-calibrated subset of LiveCodeBench v6, TTSP is the best-performing method at every verifier strength we evaluate, and the gap to the strongest baseline widens as the verifier weakens; in the no-verifier limit, TTSP yields a 35\% absolute improvement in mean@8. The gains hold across a range of verifier strengths, except when the base policy has near-zero coverage of the correct answer.