62 citations · 117 across the 5 of their papers we have counts for
5 papers
S-SimCSE: Sampled Sub-networks for Contrastive Learning of Sentence Embedding
Junlei Zhang, Zhenzhong lan
Contrastive learning has been studied for improving the performance of learning sentence embeddings. The current state-of-the-art method is the SimCSE, which takes dropout as the d…
Dynamic Resolution Network
Mingjian Zhu, Kai Han, Enhua Wu +4
Deep convolutional neural networks (CNNs) are often of sophisticated design with numerous learnable parameters for the accuracy reason. To alleviate the expensive costs of deployin…
Do Transformer Modifications Transfer Across Implementations and Applications?
Sharan Narang, Hyung Won Chung, Yi Tay +13
The research community has proposed copious modifications to the Transformer architecture since it was introduced over three years ago, relatively few of which have seen widespread…
CLUE: A Chinese Language Understanding Evaluation Benchmark
Liang Xu, Hai Hu, Xuanwei Zhang +29
The advent of natural language understanding (NLU) benchmarks for English, such as GLUE and SuperGLUE allows new NLU models to be evaluated across a diverse set of tasks. These com…
Talking-Heads Attention
Noam Shazeer, Zhenzhong Lan, Youlong Cheng +2
We introduce "talking-heads attention" - a variation on multi-head attention which includes linearprojections across the attention-heads dimension, immediately before and after the…