Dispelling the Curse of Singularities in Neural Network Optimizations

Hengjie Cao, Mengyi Chen, Yifeng Yang +11

This work investigates the optimization instability of deep neural networks from a less-explored yet insightful perspective: the emergence and amplification of singularities in the…