Gradient Equilibrium in Online Learning: Theory and Applications

Anastasios N. Angelopoulos, Michael I. Jordan, Ryan J. Tibshirani

We present a new perspective on online learning that we refer to as gradient equilibrium: a sequence of iterates achieves gradient equilibrium if the average of gradients of losses…