Qualitatively characterizing neural network optimization problems

Ian J. Goodfellow, Oriol Vinyals, Andrew M. Saxe

Training neural networks involves solving large-scale non-convex optimization problems. This task has long been believed to be extremely difficult, with fear of local minima and ot…