Learning-Rate-Free Learning: Dissecting D-Adaptation and Probabilistic Line Search

Max McGuinness

This paper explores two recent methods for learning rate optimisation in stochastic gradient descent: D-Adaptation (arXiv:2301.07733) and probabilistic line search (arXiv:1502.0284…