4 papers
Gradient Descent Provably Optimizes Over-parameterized Neural Networks
Simon S. Du, Xiyu Zhai, Barnabas Poczos +1
One of the mysteries in the success of neural networks is randomly initialized first order methods like gradient descent can achieve zero training loss even though the objective fu…
Local White Matter Architecture Defines Functional Brain Dynamics
Yo Joong Choe, Sivaraman Balakrishnan, Aarti Singh +2
Large bundles of myelinated axons, called white matter, anatomically connect disparate brain regions together and compose the structural core of the human connectome. We recently p…
Towards Understanding the Generalization Bias of Two Layer Convolutional Linear Classifiers with Gradient Descent
Yifan Wu, Barnabas Poczos, Aarti Singh
A major challenge in understanding the generalization of deep learning is to explain why (stochastic) gradient descent can exploit the network architecture to find solutions that h…
Gradient Descent Can Take Exponential Time to Escape Saddle Points
Simon S. Du, Chi Jin, Jason D. Lee +3
Although gradient descent (GD) almost always escapes saddle points asymptotically [Lee et al., 2016], this paper shows that even with fairly natural random initialization schemes a…