This paper introduces Curvature-Weighted Gradient Diversity (CWGD), a geometry-aware measure for optimization noise that reduces the asymptotic optimization error floor by up to a factor of two compar…

This paper provides the first non-vacuous generalization analysis for the Stochastic Variance Reduced Gradient (SVRG) method by establishing sharp, data-dependent algorithmic stability bounds, thereby…

The paper proposes 2FFS, a two-fidelity tree-search algorithm that efficiently identifies the best action in stochastic minimax trees by adaptively combining cheap, biased heuristic evaluations with e…

This paper shows that stochastic nonconvex optimization can be reduced to ordinary static regret minimization in online convex optimization, and establishes convergence rates for smooth and Lipschitz…

This paper proposes a data collection strategy using solver iterates to augment datasets for training generative models, improving the efficiency of the data-model-optimization loop in one-sided box-c…

This paper analyzes the poor performance of Meta-learning for Training-data Selection (MTS) and proposes that increasing the batch size and incorporating informative features can significantly improve…

The paper introduces Regularized Large Neighborhood Search (RLNS), a method that adapts the LNS heuristic into an efficient MCMC sampler for combinatorial optimization, allowing end-to-end learning wi…

This paper proves the conjecture that Local SGD outperforms Mini-batch SGD under bounded second-order heterogeneity for general convex objectives, improving the convergence guarantee and lower bounds.

TailLoR is a new parameter-efficient finetuning method that uses the singular bases of pre-trained weights to learn low-rank updates, specifically penalizing updates along dominant directions to impro…

The paper introduces TSVD, a novel framework that efficiently pre-trains LLMs by enforcing both low rank and strict weight orthonormality, achieving performance comparable to full-parameter models wit…

The paper proposes a novel online learning algorithm that achieves an interval regret bound scaling with gradient variation, providing strong theoretical guarantees for non-stationary environments.

The paper establishes tight upper and lower bounds on the statistical cost of approximate machine unlearning for smooth strongly convex losses, showing that the optimal unlearning rate depends critica…

DRIFT proposes a novel framework that efficiently optimizes LLMs for multi-turn interactions by decoupling rollout from optimization, allowing the use of weighted supervised fine-tuning to match the p…

The paper investigates applying Riemannian optimization techniques to low-rank matrix parameters for deep learning, but finds that the proposed methods do not conclusively outperform the AdamW baselin…

The paper establishes conditions for QUBO formulations of combinatorial optimization problems that guarantee valid binary and feasible local minimizers using gradient-based methods.

The paper analyzes a new class of asynchronous adaptive first-order optimization methods and proves their stochastic convergence rate is O(1/sqrt{t}) for non-convex functions.

The paper proposes S2L-PO, a framework that uses smaller, naturally diverse models as structured explorers to enhance the policy-level diversity and performance of larger language models during traini…

This paper analyzes the statistical and computational limits of learning bounded linear operators between Sobolev spaces from noisy data, and constructs a finite-resolution blockwise least-squares est…

The paper introduces Drifting Preference Optimization (DrPO), an efficient online method for preference finetuning one-step text-to-image generators that avoids complex gradient calculations and model…