This paper proposes a preconditioning layer for stable weight conditioning in LLM training.

This paper shows that stochastic nonconvex optimization can be reduced to ordinary static regret minimization in online convex optimization, and establishes convergence rates for smooth and Lipschitz…

The paper proposes FOAM, an adaptive damping method that stabilizes the Shampoo optimization algorithm by dynamically controlling damping and eigendecomposition frequency, thereby reducing staleness-i…

The paper proposes using a Physics-Informed Neural Network (PINN) residual as an efficient, physics-guided indicator to guide adaptive mesh refinement (AMR) for classical finite-difference PDE solvers…

This paper investigates the benefits of generating multiple solutions in each generation for Evolutionary Diversity Optimisation (EDO) and proposes efficient methods to achieve it.

A new method for selecting knots in Generalized Additive Models using an extension of adaptive splines and a customized Fellner-Schall scheme.

This paper presents a local computation algorithm to approximate the top eigenvector of a symmetric matrix with entries between -1 and 1, building on Swartworth and Woodruff's work.

This paper introduces structured cut sets, a novel preprocessing technique for Maximum k-Cut, and extends existing techniques from Maximum Cut. The rules are optimality-preserving and yield significan…

This paper introduces a structural theorem for the sparsifiability of real-valued codes, which generalizes both combinatorial and continuous notions of sparsification.

Introduce Deep Second-Order Stochastic Residual Method (D2SRM) for high-dimensional, Hessian-dependent fully nonlinear parabolic PDEs, establish well-posedness, and develop population-level convergenc…

The paper proposes DiReCT, a novel framework that treats data selection during LLM annealing as a constrained optimization problem based on the spectral geometry of the loss landscape, achieving state…

The paper introduces MOMENT, a framework for selecting and estimating random-effects covariance matrices and fixed-effects coefficients using moment-based methods, inducing sparsity through a positive…

The paper analyzes a new class of asynchronous adaptive first-order optimization methods and proves their stochastic convergence rate is O(1/sqrt{t}) for non-convex functions.

A new solver-free parallel spectral sparsification algorithm for weighted graphs is presented, relying on low-diameter decompositions and independent sampling, eliminating dependence on target approxi…