ArXivCSExplorer
☆☆Bookmarks🏆RSSHow to UseFAQ
Built with and by Teycir Ben Soltane•
How to Use•FAQ•GitHub•arXiv.org•
Share:

20 results for “stochastic-gradient algorithm”

CS papers only

Hybrid search: Keyword + semantic, ranked by combined score.ⓘ

Want pure semantic search? Try claim verification →

math.OCmath.NAstat.MLTheoreticalRecentJul 16, 2026

Tamed Stochastic Gradient Hamiltonian Monte Carlo

Zhuoran Wang, Ying Zhang

This paper introduces a new tamed stochastic gradient Hamiltonian Monte Carlo (tSGHMC) algorithm for sampling and optimization problems with large stochastic gradients, providing error bounds and perf…

View →
cs.LGcs.AImath.OCTheoreticalRecentJul 29, 2026

Parameter-Free Dynamic Regret for Online Convex Optimization under Heavy-Tailed Noise

Vaneet Aggarwal

This paper proposes HT-PAder, a parameter-free algorithm for online convex optimization in non-stationary environments with heavy-tailed noise, achieving an expected universal dynamic regret.

View →
cs.LGmath.OCstat.MLTheoreticalRecentJul 20, 2026

Optimizing the Preconditioner: A Black-box Online-to-Nonconvex Conversion with Static Regret Minimization Oracles

Haichen Hu, David Simchi-Levi

This paper shows that stochastic nonconvex optimization can be reduced to ordinary static regret minimization in online convex optimization, and establishes convergence rates for smooth and Lipschitz…

View →
cs.LGcs.AIRecentMay 27, 2026

Learning Theory of the SVRG: Generalization and Convergence Analysis

Yunwen Lei, Zimeng Wang, Xiaoming Yuan

This paper provides the first non-vacuous generalization analysis for the Stochastic Variance Reduced Gradient (SVRG) method by establishing sharp, data-dependent algorithmic stability bounds, thereby…

View →
cs.LGcs.ARmath.OCTheoreticalRecentJul 31, 2026

Adaptivity via a Parallel Architecture for Stochastic Gradient Methods Adaptivity via a Parallel Architecture for Stochastic Gradient Methods Adaptivity via a Parallel Architecture for Stochastic Gradient Methods

Bin Fu

A parallel framework is proposed to improve the adaptivity of static gradient methods by having processors search for appropriate iteration numbers.

View →
math.OCcs.CCcs.LGSurveyRecentOct 29, 2026

Stochastic and Noisy Oracles in Smooth Continuous Optimization

Albert S. Berahas, Liyuan Cao, Katya Scheinberg

This paper reviews models of inexact computational oracles in smooth continuous optimization and their implications for optimization algorithms.

View →
cs.AImath.OCRecentJun 1, 2026

Stochastic convergence of parallel asynchronous adaptive first-order methods

Serge Gratton, Philippe L. Toint

The paper analyzes a new class of asynchronous adaptive first-order optimization methods and proves their stochastic convergence rate is O(1/sqrt{t}) for non-convex functions.

View →
cs.LGmath.OCstat.MLTheoreticalRecentJun 29, 2026

Curvature-Weighted Gradient Diversity: A Noise Measure for Geometry-Adaptive SGD Schedules

Muhammad Hamza, Ayush Goel

This paper introduces Curvature-Weighted Gradient Diversity (CWGD), a geometry-aware measure for optimization noise that reduces the asymptotic optimization error floor by up to a factor of two compar…

View →
stat.MLcs.LGTheoreticalRecentOct 12, 2026

A Nonlinear Time-Series Theory for Constant-step SGD with Momentum

Younghun Lee, Jake Zummo, Wei Biao Wu

The paper studies the stationary behavior and distributional limits of constant-step stochastic gradient descent with momentum (SGDM) as a nonlinear time series.

View →
math.OCcs.DSTheoreticalRecentJun 15, 2026

Adaptive Proximal Methods for Weakly Convex Optimization with Unknown Parameter: Deterministic and Stochastic Guarantees

Miaolan Xie

This paper proposes a one-trial proximal algorithm for minimizing nonsmooth, nonconvex objectives with unknown weak-convexity parameter.

View →
cs.NEEmpiricalRecentJun 18, 2026

Weight Adaptation for Improving Parallel Performance of Adaptive Stochastic Natural Gradient

Yutaro Yamada, Kento Uchida, Shinichi Shirakawa

The paper proposes Weight Adaptation ASNG, a probabilistic model-based evolutionary algorithm that adapts its weight parameters to improve optimization efficiency and robustness.

View →
math.PRmath.STstat.MLTheoreticalRecentJun 24, 2026

A functional central limit theorem for kernel gradient flow and infinitesimal gradient boosting

Clément Dombry, Jean-Jil Duchamps

This paper studies the fluctuations of infinitesimal gradient boosting and establishes a functional central limit theorem.

View →
cs.LGcs.AIRecentMay 27, 2026

Stochastic Gradient Descent with Momentum is Algorithmically Stable

Yunwen Lei, Zimeng Wang, Xiaoming Yuan

This paper provides a comprehensive generalization analysis of Stochastic Gradient Descent with Momentum (SGDM) by establishing tight, on-average model stability bounds that show SGDM can generalize w…

View →
math.NAstat.MLTheoreticalRecentJul 28, 2026

Sequential Preconditioned Conjugate Gradient Method for Linear Statistical Models

Guan-Yu Chen, Dong-Yue Xie, Xi Yang, Zun-Hao Zheng

This paper proposes a randomized iterative method called Sequential Preconditioned Conjugate Gradient Method (SPCG) for large-scale linear statistical models, which significantly reduces computational…

View →
math.OCcs.LGstat.MLTheoreticalRecentJun 30, 2026

Random Reshuffling Dominates Stochastic Gradient Descent

Zijian Liu

This paper proves that Random Reshuffling in Shuffling Stochastic Gradient Descent dominates vanilla SGD in smooth convex optimization after any finite number of epochs.

View →
cs.NEEmpiricalRecentSep 18, 2026

A Confidence-Driven Evolutionary Algorithm for Noisy Optimization with Joint Chance Constraints

Enrico Halim, Hemant Singh, Tapabrata Ray

Proposes CR-EA-C, a confidence-driven evolutionary algorithm for solving noisy black-box optimization problems under joint chance constraints.

View →
econ.EMcs.LGstat.MLTheoreticalRecentJul 21, 2026

Optimizing Regret

Irene Aldridge

This paper derives the complete theory of covariance regret functional for decision making, providing insights on steepest-descent directions and boundary-optimal solutions.

View →