Dual Gauss-Newton Directions for Deep Learning

Vincent Roulet, Mathieu Blondel

Inspired by Gauss-Newton-like methods, we study the benefit of leveraging the structure of deep learning objectives, namely, the composition of a convex loss function and of a nonl…