22 citations · 107 across the 43 of their papers we have counts for
Showing stat.MLShow all
3 papers · 1 filter
stat.ML2018
Train Feedfoward Neural Network with Layer-wise Adaptive Rate via Approximating Back-matching Propagation
Huishuai Zhang, Wei Chen, Tie-Yan Liu
Stochastic gradient descent (SGD) has achieved great success in training deep neural network, where the gradient is computed through back-propagation. However, the back-propagated…
stat.ML2018
Generalization Error Bounds with Probabilistic Guarantee for SGD in Nonconvex Optimization
Yi Zhou, Yingbin Liang, Huishuai Zhang
The success of deep learning has led to a rising interest in the generalization property of the stochastic gradient descent (SGD) method, and stability is one popular approach to s…
stat.ML2018
-SGD: Optimizing ReLU Neural Networks in its Positively Scale-Invariant Space
Qi Meng, Shuxin Zheng, Huishuai Zhang +3
It is well known that neural networks with rectified linear units (ReLU) activation functions are positively scale-invariant. Conventional algorithms like stochastic gradient desce…