21 citations · 59 across the 26 of their papers we have counts for
Showing 2021Show all
2 papers · 1 filter
cs.CL2021★ 4 cited
AutoBERT-Zero: Evolving BERT Backbone from Scratch
Jiahui Gao, Hang Xu, Han Shi +5
Transformer-based pre-trained language models like BERT and its variants have recently achieved promising performance in various natural language processing (NLP) tasks. However, t…
cs.LG2021★ 13 cited
SparseBERT: Rethinking the Importance Analysis in Self-attention
Han Shi, Jiahui Gao, Xiaozhe Ren +4
Transformer-based models are popularly used in natural language processing (NLP). Its core component, self-attention, has aroused widespread interest. To understand the self-attent…