61 citations · 88 across the 13 of their papers we have counts for
6 papers · 1 filter
Deep Semi-supervised Learning with Double-Contrast of Features and Semantics
Quan Feng, Jiayu Yao, Zhison Pan +1
In recent years, the field of intelligent transportation systems (ITS) has achieved remarkable success, which is mainly due to the large amount of available annotation data. Howeve…
Provable Acceleration of Nesterov's Accelerated Gradient Method over Heavy Ball Method in Training Over-Parameterized Neural Networks
Xin Liu, Wei Tao, Wei Li +3
Due to its simplicity and efficiency, the first-order gradient method has been extensively employed in training neural networks. Although the optimization problem of the neural net…
A Convergence Analysis of Nesterov's Accelerated Gradient Method in Training Deep Linear Neural Networks
Xin Liu, Wei Tao, Zhisong Pan
Momentum methods, including heavy-ball~(HB) and Nesterov's accelerated gradient~(NAG), are widely used in training neural networks for their fast convergence. However, there is a l…
Provable Convergence of Nesterov's Accelerated Gradient Method for Over-Parameterized Neural Networks
Xin Liu, Zhisong Pan, Wei Tao
Momentum methods, such as heavy ball method~(HB) and Nesterov's accelerated gradient method~(NAG), have been widely used in training neural networks by incorporating the history of…
Gradient Descent Averaging and Primal-dual Averaging for Strongly Convex Optimization
Wei Tao, Wei Li, Zhisong Pan +1
Averaging scheme has attracted extensive attention in deep learning as well as traditional machine learning. It achieves theoretically optimal convergence and also improves the emp…
The Strength of Nesterov's Extrapolation in the Individual Convergence of Nonsmooth Optimization
W. Tao, Z. Pan, G. Wu +1
The extrapolation strategy raised by Nesterov, which can accelerate the convergence rate of gradient descent methods by orders of magnitude when dealing with smooth convex objectiv…