The Low-Rank Simplicity Bias in Deep Networks

Minyoung Huh, Hossein Mobahi, Richard Zhang +3

Modern deep neural networks are highly over-parameterized compared to the data on which they are trained, yet they often generalize remarkably well. A flurry of recent work has ask…