133 citations · 171 across the 9 of their papers we have counts for
Showing 2018Show all
3 papers · 1 filter
cs.LG2018
On the Convergence of A Class of Adam-Type Algorithms for Non-Convex Optimization
Xiangyi Chen, Sijia Liu, Ruoyu Sun +1
This paper studies a class of adaptive gradient based momentum algorithms that update the search directions and learning rates simultaneously using past gradients. This class, whic…
stat.ML2018
Adding One Neuron Can Eliminate All Bad Local Minima
Shiyu Liang, Ruoyu Sun, Jason D. Lee +1
One of the main difficulties in analyzing neural networks is the non-convexity of the loss function which may have many bad local minima. In this paper, we study the landscape of n…
cs.LG2018
Understanding the Loss Surface of Neural Networks for Binary Classification
Shiyu Liang, Ruoyu Sun, Yixuan Li +1
It is widely conjectured that the reason that training algorithms for neural networks are successful because all local minima lead to similar performance, for example, see (LeCun e…