Low-Precision Training of Large Language Models: Methods, Challenges, and Opportunities

Zhiwei Hao, Jianyuan Guo, Li Shen +6

Large language models (LLMs) have achieved impressive performance across various domains. However, the substantial hardware resources required for their training present a signific…