On stochastic gradient Langevin dynamics with dependent data streams: the fully non-convex case

Abstract

We consider the problem of sampling from a target distribution, which is \emph {not necessarily logconcave}, in the context of empirical risk minimization and stochastic optimization as presented in Raginsky et al. (2017). Non-asymptotic analysis results are established in the $L^1$-Wasserstein distance for the behaviour of Stochastic Gradient Langevin Dynamics (SGLD) algorithms. We allow the estimation of gradients to be performed even in the presence of \emph{dependent} data streams. Our convergence estimates are sharper and \emph{uniform} in the number of iterations, in contrast to those in previous studies.

References in corpus (3)

• Non-convex learning via Stochastic Gradient Langevin Dynamics: a nonasymptotic analysis
• Non-asymptotic convergence analysis for the Unadjusted Langevin Algorithm
• Convergence of Langevin MCMC in KL-divergence

Cited by in corpus (15)

• Strong error analysis for stochastic gradient descent optimization algorithms
• Full error analysis for the training of deep neural networks
• Nonasymptotic estimates for Stochastic Gradient Langevin Dynamics under local conditions in nonconvex optimization
• Wasserstein Control of Mirror Langevin Monte Carlo
• Nonasymptotic analysis of Stochastic Gradient Hamiltonian Monte Carlo under local conditions for nonconvex optimization
• Faster Convergence of Stochastic Gradient Langevin Dynamics for Non-Log-Concave Sampling
• On stochastic gradient Langevin dynamics with dependent data streams in the logconcave case
• Schr{ö}dinger-F{ö}llmer Sampler: Sampling without Ergodicity
• Projected Stochastic Gradient Langevin Algorithms for Constrained Sampling and Non-Convex Learning
• A fully data-driven approach to minimizing CVaR for portfolio of assets via SGLD with discontinuous updating
• A Decentralized Approach to Bayesian Learning
• Non-asymptotic estimates for TUSLA algorithm for non-convex learning with applications to neural networks with ReLU activation function
• Polygonal Unadjusted Langevin Algorithms: Creating stable and efficient adaptive algorithms for neural networks
• Multilevel Monte Carlo learning
• Convergence Analysis of Schr{ö}dinger-F{ö}llmer Sampler without Convexity