Stochastic Gradient Langevin Dynamics with Variance Reduction
arXiv:2102.06759 · doi:10.1109/IJCNN52387.2021.9533646
Abstract
Stochastic gradient Langevin dynamics (SGLD) has gained the attention of optimization researchers due to its global optimization properties. This paper proves an improved convergence property to local minimizers of nonconvex objective functions using SGLD accelerated by variance reductions. Moreover, we prove an ergodicity property of the SGLD scheme, which gives insights on its potential to find global minimizers of nonconvex objectives.
References in corpus (4)
- SAGA: A Fast Incremental Gradient Method With Support for Non-Strongly Convex Composite Objectives
- How to Escape Saddle Points Efficiently
- Escaping From Saddle Points --- Online Stochastic Gradient for Tensor Decomposition
- Efficiently avoiding saddle points with zero order methods: No gradients required