Adacc: An Adaptive Framework Unifying Compression and Activation Recomputation for LLM Training

Ping Chen, Zhuohong Deng, Ping Li +6

Training large language models (LLMs) is often constrained by GPU memory limitations. To alleviate memory pressure, activation recomputation and data compression have been proposed…