A functional central limit theorem for kernel gradient flow and infinitesimal gradient boosting
This paper studies the fluctuations of infinitesimal gradient boosting and establishes a functional central limit theorem.
This paper introduces a general stochastic perturbation analysis of ODEs in Banach spaces, which is a novel contribution to the field of machine learning.
Keywords
Before reading this…
Applications
- →Machine learning
- →Gradient boosting
To understand this paper, make sure you know these concepts first:
- Machine learningfind papers →
- Ordinary differential equationsfind papers →
Abstract
More Like ThisBuilding on the large-sample analysis of infinitesimal gradient boosting (Dombry and Duchamps, 2024b), we study the fluctuations of the process around its deterministic limit and establish a functional central limit theorem: the rescaled deviations converge in distribution to a Gaussian process. The analysis is carried out in a reproducing kernel Hilbert space (RKHS) naturally associated with the softmax gradient tree base learner, in which the boosting process is characterized as the solution of an autonomous ordinary differential equation (ODE). The proof rests on a general stochastic perturbation analysis of ODEs in Banach spaces, which is of independent interest: whenever a sequence of vector fields converges and satisfies a central limit theorem, so does the associated ODE solution. We first illustrate this perturbation approach in the simpler setting of kernel gradient flow, where the Gaussian limit admits an explicit characterization, and then consider the more complicated tree-based gradient boosting setting.