On the Benefit of Width for Neural Networks: Disappearance of Bad Basins

Dawei Li, Tian Ding, Ruoyu Sun

Wide networks are often believed to have a nice optimization landscape, but what rigorous results can we prove? To understand the benefit of width, it is important to identify the…