Showing 2026Show all
2 papers · 1 filter
cs.LG2026
SOLAR: A State-Driven Online Learning Rate Scheduler for LLM Pretraining
Qiulin Shang, Binyu Wang, Yongqi Qiao +3
Learning-rate (LR) scheduling plays a central role in large language model (LLM) pretraining, yet current practice still relies heavily on hand-crafted heuristics such as Warmup-Co…
cs.LG2026
QuantForge: Discovering Residual Decompositions for MXFP4 Post-Training Quantization
Qiulin Shang, Zhoutong Wu, Jie Hu +1
Four-bit post-training quantization can reduce the memory demands of large language models, but preserving accuracy under strict MXFP4 W4A4 requires coordinating several design cho…