EDGC: Entropy-driven Dynamic Gradient Compression for Efficient LLM Training

Qingao Yi, Jiaang Duan, Jun Zhang +11

Training large language models (LLMs) at scale incurs substantial communication overhead, while static gradient compression cannot adapt to gradient evolution and may degrade model…