80 citations · 158 across the 26 of their papers we have counts for
6 papers · 1 filter
NoisyTune: A Little Noise Can Help You Finetune Pretrained Language Models Better
Chuhan Wu, Fangzhao Wu, Tao Qi +2
Effectively finetuning pretrained language models (PLMs) is critical for their success in downstream tasks. However, PLMs may have risks in overfitting the pretraining tasks and da…
Fastformer: Additive Attention Can Be All You Need
Chuhan Wu, Fangzhao Wu, Tao Qi +2
Transformer is a powerful model for text understanding. However, it is inefficient due to its quadratic complexity to input sequence length. Although there are many methods on Tran…
Smart Bird: Learnable Sparse Attention for Efficient and Effective Transformer
Chuhan Wu, Fangzhao Wu, Tao Qi +4
Transformer has achieved great success in NLP. However, the quadratic complexity of the self-attention mechanism in Transformer makes it inefficient in handling long sequences. Man…
NewsBERT: Distilling Pre-trained Language Model for Intelligent News Application
Chuhan Wu, Fangzhao Wu, Yang Yu +3
Pre-trained language models (PLMs) like BERT have made great progress in NLP. News articles usually contain rich textual information, and PLMs have the potentials to enhance news t…
Improving Attention Mechanism with Query-Value Interaction
Chuhan Wu, Fangzhao Wu, Tao Qi +1
Attention mechanism has played critical roles in various state-of-the-art NLP models such as Transformer and BERT. It can be formulated as a ternary function that maps the input qu…
FedNER: Privacy-preserving Medical Named Entity Recognition with Federated Learning
Suyu Ge, Fangzhao Wu, Chuhan Wu +3
Medical named entity recognition (NER) has wide applications in intelligent healthcare. Sufficient labeled data is critical for training accurate medical NER model. However, the la…