Showing cs.LGShow all
3 papers · 1 filter
cs.LG2026
Decoupled Orthogonal Dynamics: Regularization for Deep Network Optimizers
Hao Chen, Jinghui Yuan, Hanmin Zhang
Is the standard weight decay in AdamW truly optimal? Although AdamW decouples weight decay from adaptive gradient scaling, a fundamental conflict remains: the Radial Tug-of-War. In…
cs.LG2025
Understanding the Generalization of Stochastic Gradient Adam in Learning Neural Networks
Xuan Tang, Han Zhang, Yuan Cao +1
Adam is a popular and widely used adaptive gradient method in deep learning, which has also received tremendous focus in theoretical research. However, most existing theoretical wo…
cs.LG2024
Understanding the Benefits of SimCLR Pre-Training in Two-Layer Convolutional Neural Networks
Han Zhang, Yuan Cao
SimCLR is one of the most popular contrastive learning methods for vision tasks. It pre-trains deep neural networks based on a large amount of unlabeled data by teaching the model…