Implicit Bias in Matrix Factorization and its Explicit Realization in a New Architecture

Yikun Hou, Suvrit Sra, Alp Yurtsever

Gradient descent for matrix factorization exhibits an implicit bias toward approximately low-rank solutions. While existing theories often assume the boundedness of iterates, empir…