Gradient Descent for Deep Matrix Factorization: Dynamics and Implicit Bias towards Low Rank

Hung-Hsu Chou, Carsten Gieshoff, Johannes Maly +1

In deep learning, it is common to use more network parameters than training points. In such scenarioof over-parameterization, there are usually multiple networks that achieve zero…