Data Efficacy for Language Model Training

Yalun Dai, Yangyu Huang, Xin Zhang +6

Data is fundamental to the training of language models (LM). Recent research has been dedicated to data efficiency, which aims to maximize performance by selecting a minimal or opt…