Optimal score function estimation via derivatives constraints
The paper investigates score function estimation via empirical risk minimization, focusing on score-based generative modeling and the use of Sobolev balls to prevent overfitting.
The paper introduces a new approach to score function estimation using Sobolev balls, which is novel compared to prior work.
Before reading this…
Applications
- →Generative modeling
- →Machine learning
To understand this paper, make sure you know these concepts first:
- Probability theoryfind papers →
- Functional analysisfind papers →
Abstract
More Like ThisWe consider the problem of score function estimation via empirical risk minimization. We first start with the question of inferring the score function of a probability measure $μ$ with density on the flat torus from a sample of distribution $μ$. We show that constraining the hypothesis space to a Sobolev ball is sufficient to prevent overfitting and obtaining minimax estimation rates. We then consider the problem of score function estimation in the context of score-based generative modeling. Again, under a conjecture tying the score estimation rates to the quality of the output of a score-based generative model, we obtain minimax rates for such an approach using score function estimators obtained by constraining the hypothesis class to a Sobolev ball.