This paper proposes a preconditioning layer for stable weight conditioning in LLM training.

This paper proposes a randomized iterative method called Sequential Preconditioned Conjugate Gradient Method (SPCG) for large-scale linear statistical models, which significantly reduces computational…

The paper proposes FOAM, an adaptive damping method that stabilizes the Shampoo optimization algorithm by dynamically controlling damping and eigendecomposition frequency, thereby reducing staleness-i…

This paper provides the first non-vacuous generalization analysis for the Stochastic Variance Reduced Gradient (SVRG) method by establishing sharp, data-dependent algorithmic stability bounds, thereby…

This paper provides a comprehensive generalization analysis of Stochastic Gradient Descent with Momentum (SGDM) by establishing tight, on-average model stability bounds that show SGDM can generalize w…

This paper shows that stochastic nonconvex optimization can be reduced to ordinary static regret minimization in online convex optimization, and establishes convergence rates for smooth and Lipschitz…

This paper proposes methods to make computationally expensive Caputo-based optimization more viable, while preserving its memory structure.

This paper proposes CUTh-Solver, a GPU-accelerated Preconditioned Conjugate Gradient (PCG)-based sparse solver framework for high-resolution 3D IC thermal simulation, achieving significant speedup ove…

This paper proposes a data collection strategy using solver iterates to augment datasets for training generative models, improving the efficiency of the data-model-optimization loop in one-sided box-c…

The paper analyzes a new class of asynchronous adaptive first-order optimization methods and proves their stochastic convergence rate is O(1/sqrt{t}) for non-convex functions.

This paper investigates the limitations of polyconvex constitutive modeling, showing that while theoretically appealing, it can impose overly restrictive constraints and perform poorly in reproducing…

This paper introduces DDC, a Dead-Direction Conditioner that keeps a deep network's optimization on the symmetry quotient by conditioning the optimizer's state in the orbit decomposition of a $G$-inva…

This paper proposes a unified sparse background-grid framework for the Material Point Method (MPM), significantly reducing computational time and memory usage in large-scale simulations where the mate…

The paper proposes using a Physics-Informed Neural Network (PINN) residual as an efficient, physics-guided indicator to guide adaptive mesh refinement (AMR) for classical finite-difference PDE solvers…

This paper introduces the Distributed Truncated Spectral Transform (DTST) for Fourier Neural Operators (FNOs), achieving significant speedups in distributed computing.

The paper proposes using pseudo-sensitivities, derived from adjoint sensitivity fields, as an optimal conditioning signal in a Bernoulli flow-matching framework to significantly improve the out-of-dis…

This paper proves the conjecture that Local SGD outperforms Mini-batch SGD under bounded second-order heterogeneity for general convex objectives, improving the convergence guarantee and lower bounds.

This paper establishes a direct connection between Approximate Message Passing (AMP) and the Convex Gaussian Min-max Theorem (CGMT) for regularized linear regression and M-estimation.