A Fourier perspective on the learning dynamics of neural networks: from sample complexities to mechanistic insights

Fabiola Ricci, Claudia Merger, Sebastian Goldt

Neural networks trained with gradient-based methods exhibit a strong simplicity bias: they learn simpler statistical features of their data before moving to more complex features.…