AdamPM: Memory-Efficient LLM Optimization via Moment Pooling

Xiaowei Ji, Lishan Xu, Jianing An +3

Adam is widely used for training large language models (LLMs) owing to its strong optimization performance and training stability, but its element-wise estimation for the first and…