2 citations · 3 across the 8 of their papers we have counts for
Showing 2025Show all
3 papers · 1 filter
cs.LG2025
Low-Precision Training of Large Language Models: Methods, Challenges, and Opportunities
Zhiwei Hao, Jianyuan Guo, Li Shen +6
Large language models (LLMs) have achieved impressive performance across various domains. However, the substantial hardware resources required for their training present a signific…
cs.LG2025
SeWA: Selective Weight Average via Probabilistic Masking
Peng Wang, Shengchao Hu, Zerui Tao +5
Weight averaging has become a standard technique for enhancing model performance. However, methods such as Stochastic Weight Averaging (SWA) and Latest Weight Averaging (LAWA) ofte…
cs.LG2025
AdaGC: Enhancing LLM Pretraining Stability via Adaptive Gradient Clipping
Guoxia Wang, Shuai Li, Congliang Chen +5
Loss spikes remain a persistent obstacle in large-scale language model pretraining. While previous research has attempted to identify the root cause of loss spikes by investigating…