2 papers
cs.LG2026
PRAC: Principal-Random Subspace for LLM Activation Compression and Memory-Efficient Training
Yanyi Li, Yimu Zhang, Cong Fang
Activations have become the primary memory bottleneck in large-batch LLM training. However, existing compression methods fail to exploit the spectral structure of activations, resu…
cs.LG2025
AdaPM: a Partial Momentum Algorithm for LLM Training
Yimu Zhang, Yuanshi Liu, Cong Fang
In the training of large language models, momentum is widely used and often demonstrated to achieve significant acceleration. However, storing momentum typically presents memory ch…