Adaptive Proximal Methods for Weakly Convex Optimization with Unknown Parameter: Deterministic and Stochastic Guarantees
This paper proposes a one-trial proximal algorithm for minimizing nonsmooth, nonconvex objectives with unknown weak-convexity parameter.
The paper introduces a novel adaptive proximal algorithm that can handle unknown weak-convexity parameters and achieve good iteration complexity in both deterministic and stochastic settings.
Before reading this…
Applications
- →learning
- →signal recovery
To understand this paper, make sure you know these concepts first:
- proximal algorithmsfind papers →
- weak-convexityfind papers →
Abstract
More Like ThisMany nonsmooth, nonconvex objectives in learning and signal recovery are $ρ$-weakly convex. We minimize such a function in deterministic and stochastic settings when the weak-convexity parameter $ρ$ is unknown. The objective is not required to be globally Lipschitz continuous or smooth. We propose the Adaptive Prox-Guided Scheme (APS), a one-trial proximal algorithm that adapts the proximal parameter online and bidirectionally through a descent test, allowing it to exploit favorable local structure. In the deterministic setting, APS obtains an $O(\varepsilon^{-2})$ iteration complexity for producing an $\varepsilon$-subgradient stationary point. In the stochastic setting, APS achieves a high-probability $O(\varepsilon^{-2})$ iteration bound for driving the Moreau-envelope gradient below $\varepsilon$. This result holds under deliberately weak oracle assumptions: the function-difference estimates may be biased and heavy-tailed, and the stochastic proximal oracle need only be sufficiently accurate with constant probability when the proximal parameter lies below $1/(2ρ)$ (unknown to the algorithm), and can be arbitrary otherwise.