Variable Metric Stochastic Approximation Theory
arXiv:0908.3529
Abstract
We provide a variable metric stochastic approximation theory. In doing so, we provide a convergence theory for a large class of online variable metric methods including the recently introduced online versions of the BFGS algorithm and its limited-memory LBFGS variant. We also discuss the implications of our results for learning from expert advice.
Correctment of theorem 3.4. from AISTATS 2009 article
Cited by in corpus (4)
- A Stochastic Gradient Method with an Exponential Convergence Rate for Finite Training Sets
- Global Convergence of Online Limited Memory BFGS
- Fast and Furious Convergence: Stochastic Second Order Methods under Interpolation
- Dual Stochastic Natural Gradient Descent and convergence of interior half-space gradient approximations