10 citations · 12 across the 5 of their papers we have counts for
Showing cs.CLShow all
3 papers · 1 filter
cs.CL2021
SAS: Self-Augmentation Strategy for Language Model Pre-training
Yifei Xu, Jingqiao Zhang, Ru He +4
The core of self-supervised learning for pre-training language models includes pre-training task design as well as appropriate data augmentation. Most data augmentations in languag…
cs.CL2020★ 10 cited
Progressively Stacking 2.0: A Multi-stage Layerwise Training Method for BERT Training Speedup
Cheng Yang, Shengnan Wang, Chao Yang +3
Pre-trained language models, such as BERT, have achieved significant accuracy gain in many natural language processing tasks. Despite its effectiveness, the huge number of paramete…
cs.CL2020
CoRe: An Efficient Coarse-refined Training Framework for BERT
Cheng Yang, Shengnan Wang, Yuechuan Li +4
In recent years, BERT has made significant breakthroughs on many natural language processing tasks and attracted great attentions. Despite its accuracy gains, the BERT model genera…