TRANSOM: An Efficient Fault-Tolerant System for Training LLMs

Baodong Wu, Lei Xia, Qingping Li +6

Large language models (LLMs) with hundreds of billions or trillions of parameters, represented by chatGPT, have achieved profound impact on various fields. However, training LLMs w…