1 citations · 1 across the 9 of their papers we have counts for
Showing 2026Show all
2 papers · 1 filter
cs.LG2026
Optimal low-rank stochastic gradient estimation for LLM training
Zehao Li, Tao Ren, Zishi Zhang +2
Large language model (LLM) training is often bottlenecked by memory constraints and stochastic gradient noise in extremely high-dimensional parameter spaces. Motivated by empirical…
cs.LG2026
Omni-Masked Gradient Descent: Memory-Efficient Optimization via Mask Traversal with Improved Convergence
Hui Yang, Tao Ren, Jinyang Jiang +2
Memory-efficient optimization methods have recently gained increasing attention for scaling full-parameter training of large language models under the GPU-memory bottleneck. Existi…