Simplicity Bias in 1-Hidden Layer Neural Networks

Depen Morwani, Jatin Batra, Prateek Jain +1

Recent works have demonstrated that neural networks exhibit extreme simplicity bias(SB). That is, they learn only the simplest features to solve a task at hand, even in the presenc…