The paper introduces GO-MUON, a method for exact muon's polar update using data-dependent geometry for spectral oracles, and shows its raw update solves weighted spectral oracles.
Proposes GO-MUON for exact muon's polar update and shows its raw update solves weighted spectral oracles
Before reading this…
Applications
- →Machine learning
- →Optimization
To understand this paper, make sure you know these concepts first:
- Understanding of spectral oracles, muon's polar update, and optimization algorithmsfind papers →
Abstract
More Like ThisMuon's polar update is exact for an unweighted spectral geometry. We introduce GO-MUON, which uses a matched data-dependent geometry and reuses it across several optimization steps. Conditioned on any positive-definite left and right maps, its raw update exactly solves the corresponding weighted spectral oracle; this statement is independent of how the maps are estimated or how recently they were refreshed. For softmax cross-entropy, we quantify when the observed-label backward factor approaches the model Fisher and generalized Gauss--Newton factor. We also show that four-step refresh nearly preserves the tracking delay of slowly changing geometry while increasing stationary factor noise, making lazy geometry a compute--statistics tradeoff rather than a denoising mechanism.