A Dynamical View on Optimization Algorithms of Overparameterized Neural Networks

Zhiqi Bu, Shiyun Xu, Kan Chen

When equipped with efficient optimization algorithms, the over-parameterized neural networks have demonstrated high level of performance even though the loss function is non-convex…