Gradient descent in Gaussian random fields as a toy model for high-dimensional optimisation in deep learning

Mariano Chouza, Stephen Roberts, Stefan Zohren

In this paper we model the loss function of high-dimensional optimization problems by a Gaussian random field, or equivalently a Gaussian process. Our aim is to study gradient desc…