Dimension-free convergence rates for gradient Langevin dynamics in RKHS
arXiv:2003.00306
Abstract
Gradient Langevin dynamics (GLD) and stochastic GLD (SGLD) have attracted considerable attention lately, as a way to provide convergence guarantees in a non-convex setting. However, the known rates grow exponentially with the dimension of the space. In this work, we provide a convergence analysis of GLD and SGLD when the optimization space is an infinite dimensional Hilbert space. More precisely, we derive non-asymptotic, dimension-free convergence rates for GLD/SGLD when performing regularized non-convex optimization in a reproducing kernel Hilbert space. Amongst others, the convergence analysis relies on the properties of a stochastic differential equation, its discrete time Galerkin approximation and the geometric ergodicity of the associated Markov chains.
References in corpus (15)
- Adam: A Method for Stochastic Optimization
- ADADELTA: An Adaptive Learning Rate Method
- Reproducing kernel Hilbert spaces of Gaussian priors
- Non-convex learning via Stochastic Gradient Langevin Dynamics: a nonasymptotic analysis
- Optimal Error Estimates of Galerkin Finite Element Methods for Stochastic Partial Differential Equations with Multiplicative Noise
- Weak convergence for a spatial approximation of the nonlinear stochastic heat equation
- Non-asymptotic convergence analysis for the Unadjusted Langevin Algorithm
- On a generalization of the preconditioned Crank-Nicolson Metropolis algorithm
- A Hitting Time Analysis of Stochastic Gradient Langevin Dynamics
- A simple proof of the Gaussian correlation conjecture extended to multivariate gamma distributions
- Duality in refined Sobolev-Malliavin spaces and weak approximations of SPDE
- A full-discrete exponential Euler approximation of invariant measure for parabolic stochastic partial differential equations
- Lower estimates of transition densities and bounds on exponential ergodicity for stochastic PDE's
- High-order integrator for sampling the invariant distribution of a class of parabolic SPDEs with additive space-time noise
- Learning Tensors in Reproducing Kernel Hilbert Spaces with Multilinear Spectral Penalties