Unified Theory of Adaptive Variance Reduction

Aleksandr Shestakov, Valery Parfenov, Aleksandr Beznosikov

Variance reduction is a family of powerful mechanisms for stochastic optimization that appears to be helpful in many machine learning tasks. It is based on estimating the exact gra…