7 papers · 1 filter
Convergence of Practical Muon with Finite Newton-Schulz Iterations and Nesterov Momentum
Hanyng Peng, Hui Wang, Yue Yu
Practical Muon maintains momentum and performs a small, fixed number of Newton--Schulz iterations separately for each parameter matrix, often with a Nesterov correction. We analyze…
Symphony-MoE: Harmonizing Disparate Pre-trained Models into a Coherent Mixture-of-Experts
Qi Wang, Hanyang Peng, Yue Yu
Mixture-of-Experts (MoE) models enable scalable performance by activating large parameter sets sparsely, minimizing computational overhead. To mitigate the prohibitive cost of trai…
GEPO: Group Expectation Policy Optimization for Stable Heterogeneous Reinforcement Learning
Han Zhang, Ruibin Zheng, Zexuan Yi +16
As single-center computing approaches power constraints, decentralized training becomes essential. However, traditional Reinforcement Learning (RL) methods, crucial for enhancing l…
SoftSignSGD(S3): An Enhanced Optimizer for Practical DNN Training and Loss Spikes Minimization Beyond Adam
Hanyang Peng, Shuang Qin, Yue Yu +3
Adam has proven remarkable successful in training deep neural networks, but the mechanisms underlying its empirical successes and limitations remain underexplored. In this study, w…
Simple Convergence Proof of Adam From a Sign-like Descent Perspective
Hanyang Peng, Shuang Qin, Yue Yu +3
Adam is widely recognized as one of the most effective optimizers for training deep neural networks (DNNs). Despite its remarkable empirical success, its theoretical convergence an…
Correcting Large Language Model Behavior via Influence Function
Han Zhang, Zhuo Zhang, Yi Zhang +8
Recent advancements in AI alignment techniques have significantly improved the alignment of large language models (LLMs) with static human preferences. However, the dynamic nature…
Online Self-Preferring Language Models
Yuanzhao Zhai, Zhuo Zhang, Kele Xu +6
Aligning with human preference datasets has been critical to the success of large language models (LLMs). Reinforcement learning from human feedback (RLHF) employs a costly reward…