Efficient Training of Large Language Models on Distributed Infrastructures: A Survey

Jiangfei Duan, Shuo Zhang, Zerui Wang +13

Large Language Models (LLMs) like GPT and LLaMA are revolutionizing the AI industry with their sophisticated capabilities. Training these models requires vast GPU clusters and sign…