Incorporating the Barzilai-Borwein Adaptive Step Size into Sugradient Methods for Deep Network Training

Antonio Robles-Kelly, Asef Nazari

In this paper, we incorporate the Barzilai-Borwein step size into gradient descent methods used to train deep networks. This allows us to adapt the learning rate using a two-point…