Showing cs.LGShow all
3 papers · 1 filter
cs.LG2026
SOLAR: A State-Driven Online Learning Rate Scheduler for LLM Pretraining
Qiulin Shang, Binyu Wang, Yongqi Qiao +3
Learning-rate (LR) scheduling plays a central role in large language model (LLM) pretraining, yet current practice still relies heavily on hand-crafted heuristics such as Warmup-Co…
cs.LG2026
QuantForge: Discovering Residual Decompositions for MXFP4 Post-Training Quantization
Qiulin Shang, Zhoutong Wu, Jie Hu +1
Four-bit post-training quantization can reduce the memory demands of large language models, but preserving accuracy under strict MXFP4 W4A4 requires coordinating several design cho…
cs.LG2025
Scalable Parameter and Memory Efficient Pretraining for LLM: Recent Algorithmic Advances and Benchmarking
Athanasios Glentis, Jiaxiang Li, Qiulin Shang +4
Fueled by their remarkable ability to tackle diverse tasks across multiple domains, large language models (LLMs) have grown at an unprecedented rate, with some recent models contai…