Preparing Lessons for Progressive Training on Language Models

Yu Pan, Ye Yuan, Yichun Yin +6

The rapid progress of Transformers in artificial intelligence has come at the cost of increased resource consumption and greenhouse gas emissions due to growing model sizes. Prior…