Gradient flow in parameter space is equivalent to linear interpolation in output space

Thomas Chen, Patrícia Muñoz Ewald

We prove that the standard gradient flow in parameter space that underlies many training algorithms in deep learning can be continuously deformed into an adapted gradient flow whic…