Rethinking Benign Overfitting in Two-Layer Neural Networks

Ruichen Xu, Kexin Chen

Recent theoretical studies (Kou et al., 2023; Cao et al., 2022) have revealed a sharp phase transition from benign to harmful overfitting when the noise-to-feature ratio exceeds a…