A Teacher-Student Perspective on the Dynamics of Learning Near the Optimal Point

Carlos Couto, José Mourão, Mário A. T. Figueiredo +1

Near an optimal learning point of a neural network, the learning performance of gradient descent dynamics is dictated by the Hessian matrix of the loss function with respect to the…