Enhancing Memory Efficiency in Large Language Model Training Through Chronos-aware Pipeline Parallelism

Xinyuan Lin, Chenlu Li, Zongle Huang +5

Larger model sizes and longer sequence lengths have empowered the Large Language Model (LLM) to achieve outstanding performance across various domains. However, this progress bring…