4 papers
Beyond -norm and -norm: A Curvature-Inspired -Norm Scheme for Deep Neural Networks
Jianhao Xu, Zhuang Yang
The existing optimizers for deep neural networks (DNNs) typically rely on either the norm or the norm, resulting in optimizers that do not adapt well to subs…
Explainable Learning Rate Regimes for Stochastic Optimization
Zhuang Yang
Modern machine learning is trained by stochastic gradient descent (SGD), whose performance critically depends on how the learning rate (LR) is adjusted and decreased over time. Yet…
Adaptive Diffusion Policy Optimization for Robotic Manipulation
Huiyun Jiang, Zhuang Yang
Recent studies have shown the great potential of diffusion models in improving reinforcement learning (RL) by modeling complex policies, expressing a high degree of multi-modality,…
Fast Stochastic Policy Gradient: Negative Momentum for Reinforcement Learning
Haobin Zhang, Zhuang Yang
Stochastic optimization algorithms, particularly stochastic policy gradient (SPG), report significant success in reinforcement learning (RL). Nevertheless, up to now, that how to s…