623 citations · 1.9k across the 48 of their papers we have counts for
Showing 2021 · cs.CLShow all
3 papers · 2 filters
cs.CL2021★ 80 cited
Fastformer: Additive Attention Can Be All You Need
Chuhan Wu, Fangzhao Wu, Tao Qi +2
Transformer is a powerful model for text understanding. However, it is inefficient due to its quadratic complexity to input sequence length. Although there are many methods on Tran…
cs.CL2021★ 2 cited
Smart Bird: Learnable Sparse Attention for Efficient and Effective Transformer
Chuhan Wu, Fangzhao Wu, Tao Qi +4
Transformer has achieved great success in NLP. However, the quadratic complexity of the self-attention mechanism in Transformer makes it inefficient in handling long sequences. Man…
cs.CL2021★ 2 cited
NewsBERT: Distilling Pre-trained Language Model for Intelligent News Application
Chuhan Wu, Fangzhao Wu, Yang Yu +3
Pre-trained language models (PLMs) like BERT have made great progress in NLP. News articles usually contain rich textual information, and PLMs have the potentials to enhance news t…