2 papers
cs.LG2026
Memory-Efficient LLM Pretraining via Minimalist Optimizer Design
Athanasios Glentis, Jiaxiang Li, Andi Han +1
Training large language models (LLMs) relies on adaptive optimizers such as Adam, which introduce extra operations and require significantly more memory to maintain first- and seco…
cs.LG2025
Scalable Parameter and Memory Efficient Pretraining for LLM: Recent Algorithmic Advances and Benchmarking
Athanasios Glentis, Jiaxiang Li, Qiulin Shang +4
Fueled by their remarkable ability to tackle diverse tasks across multiple domains, large language models (LLMs) have grown at an unprecedented rate, with some recent models contai…