Semi-flat minima and saddle points by embedding neural networks to overparameterization

Kenji Fukumizu, Shoichiro Yamaguchi, Yoh-ichi Mototake +1

We theoretically study the landscape of the training error for neural networks in overparameterized cases. We consider three basic methods for embedding a network into a wider one…