On the Benefits of Weight Normalization for Overparameterized Matrix Sensing

Yudong Wei, Liang Zhang, Bingcong Li +1

While normalization techniques are widely used in deep learning, their theoretical understanding remains relatively limited. In this work, we establish the benefits of (generalized…