13 citations · 23 across the 5 of their papers we have counts for
4 papers · 1 filter
Improving Contrastive Learning of Sentence Embeddings with Case-Augmented Positives and Retrieved Negatives
Wei Wang, Liangzhu Ge, Jingqiao Zhang +1
Following SimCSE, contrastive learning based methods have achieved the state-of-the-art (SOTA) performance in learning sentence embeddings. However, the unsupervised contrastive le…
SAS: Self-Augmentation Strategy for Language Model Pre-training
Yifei Xu, Jingqiao Zhang, Ru He +4
The core of self-supervised learning for pre-training language models includes pre-training task design as well as appropriate data augmentation. Most data augmentations in languag…
Progressively Stacking 2.0: A Multi-stage Layerwise Training Method for BERT Training Speedup
Cheng Yang, Shengnan Wang, Chao Yang +3
Pre-trained language models, such as BERT, have achieved significant accuracy gain in many natural language processing tasks. Despite its effectiveness, the huge number of paramete…
CoRe: An Efficient Coarse-refined Training Framework for BERT
Cheng Yang, Shengnan Wang, Yuechuan Li +4
In recent years, BERT has made significant breakthroughs on many natural language processing tasks and attracted great attentions. Despite its accuracy gains, the BERT model genera…