Complexities of Armijo-like algorithms in Deep Learning context

Bensaid Bilel

The classical Armijo backtracking algorithm achieves the optimal complexity for smooth functions like gradient descent but without any hyperparameter tuning. However, the smoothnes…