13 citations · 46 across the 21 of their papers we have counts for
Showing 2025Show all
2 papers · 1 filter
cs.LG2025
Towards Efficient Optimizer Design for LLM via Structured Fisher Approximation with a Low-Rank Extension
Wenbo Gong, Meyer Scetbon, Chao Ma +1
Designing efficient optimizers for large language models (LLMs) with low-memory requirements and fast convergence is an important and challenging problem. This paper makes a step t…
cs.LG2025
Gradient Multi-Normalization for Stateless and Scalable LLM Training
Meyer Scetbon, Chao Ma, Wenbo Gong +1
Training large language models (LLMs) typically relies on adaptive optimizers like Adam (Kingma & Ba, 2015) which store additional state information to accelerate convergence but i…