61 citations · 750 across the 66 of their papers we have counts for
Showing 2023Show all
3 papers · 1 filter
math.OC2023★ 1 cited
Generalized-Smooth Nonconvex Optimization is As Efficient As Smooth Nonconvex Optimization
Ziyi Chen, Yi Zhou, Yingbin Liang +1
Various optimal gradient-based algorithms have been developed for smooth nonconvex optimization. However, many nonconvex machine learning problems do not belong to the class of smo…
cs.LG2023
Pruning Before Training May Improve Generalization, Provably
Hongru Yang, Yingbin Liang, Xiaojie Guo +2
It has been observed in practice that applying pruning-at-initialization methods to neural networks and training the sparsified networks can not only retain the testing performance…
cs.LG2023
Neural Networks with Sparse Activation Induced by Large Bias: Tighter Analysis with Bias-Generalized NTK
Hongru Yang, Ziyu Jiang, Ruizhe Zhang +2
We study training one-hidden-layer ReLU networks in the neural tangent kernel (NTK) regime, where the networks' biases are initialized to some constant rather than zero. We prove t…