2 papers
cs.LG2025
SoftSignSGD(S3): An Enhanced Optimizer for Practical DNN Training and Loss Spikes Minimization Beyond Adam
Hanyang Peng, Shuang Qin, Yue Yu +3
Adam has proven remarkable successful in training deep neural networks, but the mechanisms underlying its empirical successes and limitations remain underexplored. In this study, w…
cs.LG2025
Simple Convergence Proof of Adam From a Sign-like Descent Perspective
Hanyang Peng, Shuang Qin, Yue Yu +3
Adam is widely recognized as one of the most effective optimizers for training deep neural networks (DNNs). Despite its remarkable empirical success, its theoretical convergence an…