4 citations · 6 across the 5 of their papers we have counts for
4 papers · 1 filter
Beyond -norm and -norm: A Curvature-Inspired -Norm Scheme for Deep Neural Networks
Jianhao Xu, Zhuang Yang
The existing optimizers for deep neural networks (DNNs) typically rely on either the norm or the norm, resulting in optimizers that do not adapt well to subs…
Explainable Learning Rate Regimes for Stochastic Optimization
Zhuang Yang
Modern machine learning is trained by stochastic gradient descent (SGD), whose performance critically depends on how the learning rate (LR) is adjusted and decreased over time. Yet…
Fast Stochastic Policy Gradient: Negative Momentum for Reinforcement Learning
Haobin Zhang, Zhuang Yang
Stochastic optimization algorithms, particularly stochastic policy gradient (SPG), report significant success in reinforcement learning (RL). Nevertheless, up to now, that how to s…
Accelerating Mini-batch SARAH by Step Size Rules
Zhuang Yang, Zengping Chen, Cheng Wang
StochAstic Recursive grAdient algoritHm (SARAH), originally proposed for convex optimization and also proven to be effective for general nonconvex optimization, has received great…