Modify Training Directions in Function Space to Reduce Generalization Error

Yi Yu, Wenlian Lu, Boyu Chen

We propose theoretical analyses of a modified natural gradient descent method in the neural network function space based on the eigendecompositions of neural tangent kernel and Fis…