EDGC: Entropy-driven Dynamic Gradient Compression for Efficient LLM Training

Qingao Yi, Jiaang Duan, Hanwen Hu +10

Training large language models (LLMs) poses significant challenges regarding computational resources and memory capacity. Although distributed training techniques help mitigate the…