Non-Uniform Smoothness for Gradient Descent

Albert S. Berahas, Lindon Roberts, Fred Roosta

The analysis of gradient descent-type methods typically relies on the Lipschitz continuity of the objective gradient. This generally requires an expensive hyperparameter tuning pro…