LazyHMC: Hamiltonian Monte Carlo Simulation for Lazy, Infinite Dimensional Probabilistic Programs
This paper develops new formulations of Hamiltonian Monte Carlo (HMC) for infinite-dimensional parameter spaces in Haskell using lazy evaluation, providing analysis for automatic differentiation and Monte Carlo method.
Provides theoretical foundations for using HMC with infinite-dimensional and lazy programs
Keywords
Before reading this…
Applications
- →Probabilistic programming
- →Stochastic processes
- →Non-parametric Bayesian models
To understand this paper, make sure you know these concepts first:
- Probabilistic programmingfind papers →
- Hamiltonian Monte Carlofind papers →
- Lazy evaluationfind papers →
Abstract
More Like ThisHamiltonian Monte Carlo (HMC) is a successful generic inference method in probabilistic programming, but in its ordinary formulation it needs gradients and finite-dimensional parameter spaces. In Haskell, lazy evaluation lets probabilistic programs express stochastic processes and other non-parametric Bayesian models over implicit infinite-dimensional spaces. This paper develops new formulations of gradient-based HMC for this infinite-dimensional setting, via lazy evaluation. For automatic differentiation, we provide an analysis based on a new notion of "piecewise analytic under cylindrical analytic partition" (PACAP), to show that even if a program is infinite-dimensional and defined lazily, the gradient of the likelihood function is finitely supported. For the Monte Carlo method itself, we develop several HMC variants and a No-U-Turn Sampler that operate over the infinite-dimensional parameter space but are still productive because of lazy evaluation. Experiments cover Gaussian mixture clustering, random walks, and piecewise-constant regression with Poisson-process changepoints.