315 citations · 1.5k across the 58 of their papers we have counts for
18 papers · 1 filter
NoisyTune: A Little Noise Can Help You Finetune Pretrained Language Models Better
Chuhan Wu, Fangzhao Wu, Tao Qi +2
Effectively finetuning pretrained language models (PLMs) is critical for their success in downstream tasks. However, PLMs may have risks in overfitting the pretraining tasks and da…
A New Sentence Extraction Strategy for Unsupervised Extractive Summarization Methods
Dehao Tao, Yingzhu Xiong, Zhongliang Yang +1
In recent years, text summarization methods have attracted much attention again thanks to the researches on neural network models. Most of the current text summarization methods ba…
Fastformer: Additive Attention Can Be All You Need
Chuhan Wu, Fangzhao Wu, Tao Qi +2
Transformer is a powerful model for text understanding. However, it is inefficient due to its quadratic complexity to input sequence length. Although there are many methods on Tran…
Smart Bird: Learnable Sparse Attention for Efficient and Effective Transformer
Chuhan Wu, Fangzhao Wu, Tao Qi +4
Transformer has achieved great success in NLP. However, the quadratic complexity of the self-attention mechanism in Transformer makes it inefficient in handling long sequences. Man…
Provably Secure Generative Linguistic Steganography
Siyu Zhang, Zhongliang Yang, Jinshuai Yang +1
Generative linguistic steganography mainly utilized language models and applied steganographic sampling (stegosampling) to generate high-security steganographic text (stegotext). H…
One Teacher is Enough? Pre-trained Language Model Distillation from Multiple Teachers
Chuhan Wu, Fangzhao Wu, Yongfeng Huang
Pre-trained language models (PLMs) achieve great success in NLP. However, their huge model sizes hinder their applications in many practical systems. Knowledge distillation is a po…
Hi-Transformer: Hierarchical Interactive Transformer for Efficient and Effective Long Document Modeling
Chuhan Wu, Fangzhao Wu, Tao Qi +1
Transformer is important for text modeling. However, it has difficulty in handling long documents due to the quadratic complexity with input text length. In order to handle this pr…
Distribution Matching for Rationalization
Yongfeng Huang, Yujun Chen, Yulun Du +1
The task of rationalization aims to extract pieces of input text as rationales to justify neural network predictions on text classification tasks. By definition, rationales represe…
NewsBERT: Distilling Pre-trained Language Model for Intelligent News Application
Chuhan Wu, Fangzhao Wu, Yang Yu +3
Pre-trained language models (PLMs) like BERT have made great progress in NLP. News articles usually contain rich textual information, and PLMs have the potentials to enhance news t…
Improving Attention Mechanism with Query-Value Interaction
Chuhan Wu, Fangzhao Wu, Tao Qi +1
Attention mechanism has played critical roles in various state-of-the-art NLP models such as Transformer and BERT. It can be formulated as a ternary function that maps the input qu…
DA-Transformer: Distance-aware Transformer
Chuhan Wu, Fangzhao Wu, Yongfeng Huang
Transformer has achieved great success in the NLP field by composing various advanced models like BERT and GPT. However, Transformer and its existing variants may not be optimal in…
Graph-Stega: Semantic Controllable Steganographic Text Generation Guided by Knowledge Graph
Zhongliang Yang, Baitao Gong, Yamin Li +3
Most of the existing text generative steganographic methods are based on coding the conditional probability distribution of each word during the generation process, and then select…
FedNER: Privacy-preserving Medical Named Entity Recognition with Federated Learning
Suyu Ge, Fangzhao Wu, Chuhan Wu +3
Medical named entity recognition (NER) has wide applications in intelligent healthcare. Sufficient labeled data is critical for training accurate medical NER model. However, the la…
Neural News Recommendation with Attentive Multi-View Learning
Chuhan Wu, Fangzhao Wu, Mingxiao An +3
Personalized news recommendation is very important for online news platforms to help users find interested news and improve user experience. News and user representation learning i…
Neural Chinese Named Entity Recognition via CNN-LSTM-CRF and Joint Training with Word Segmentation
Fangzhao Wu, Junxin Liu, Chuhan Wu +2
Chinese named entity recognition (CNER) is an important task in Chinese natural language processing field. However, CNER is very challenging since Chinese entity names are highly c…
Neural Chinese Word Segmentation with Lexicon and Unlabeled Data via Posterior Regularization
Junxin Liu, Fangzhao Wu, Chuhan Wu +2
Existing methods for CWS usually rely on a large number of labeled sentences to train word segmentation models, which are expensive and time-consuming to annotate. Luckily, the unl…
Neural Chinese Word Segmentation with Dictionary Knowledge
Junxin Liu, Fangzhao Wu, Chuhan Wu +2
Chinese word segmentation (CWS) is an important task for Chinese NLP. Recently, many neural network based methods have been proposed for CWS. However, these methods require a large…
Clinical Assistant Diagnosis for Electronic Medical Record Based on Convolutional Neural Network
Zhongliang Yang, Yongfeng Huang, Yiran Jiang +3
Automatically extracting useful information from electronic medical records along with conducting disease diagnoses is a promising task for both clinical decision support(CDS) and…