Pre-Trained Models: Past, Present and Future

Xu Han, Zhengyan Zhang, Ning Ding +21

Large-scale pre-trained models (PTMs) such as BERT and GPT have recently achieved great success and become a milestone in the field of artificial intelligence (AI). Owing to sophis…