Dive into Big Model Training

Qinghua Liu, Yuxiang Jiang

The increasing scale of model size and continuous improvement of performance herald the arrival of the Big Model era. In this report, we explore what and how the big model training…