How Does Overparameterization Affect Features?

Ahmet Cagri Duzgun, Samy Jelassi, Yuanzhi Li

Overparameterization, the condition where models have more parameters than necessary to fit their training loss, is a crucial factor for the success of deep learning. However, the…