Showing 2025Show all
2 papers · 1 filter
cs.LG2025
A Memory Efficient Randomized Subspace Optimization Method for Training Large Language Models
Yiming Chen, Yuan Zhang, Yin Liu +2
The memory challenges associated with training Large Language Models (LLMs) have become a critical concern, particularly when using the Adam optimizer. To address this issue, numer…
cs.LG2025
GWT: Scalable Optimizer State Compression for Large Language Model Training
Ziqing Wen, Ping Luo, Jiahuan Wang +4
Training large language models (LLMs) requires substantial memory, a significant fraction of which is consumed by the moment states maintained by adaptive optimizers such as Adam.…