134 citations · 189 across the 22 of their papers we have counts for
Showing cs.LGShow all
2 papers · 1 filter
cs.LG2024★ 1 cited
Preparing Lessons for Progressive Training on Language Models
Yu Pan, Ye Yuan, Yichun Yin +6
The rapid progress of Transformers in artificial intelligence has come at the cost of increased resource consumption and greenhouse gas emissions due to growing model sizes. Prior…
cs.LG2023★ 4 cited
Reusing Pretrained Models by Multi-linear Operators for Efficient Training
Yu Pan, Ye Yuan, Yichun Yin +4
Training large models from scratch usually costs a substantial amount of resources. Towards this problem, recent studies such as bert2BERT and LiGO have reused small pretrained mod…