1k citations · 2.7k across the 61 of their papers we have counts for
9 papers · 1 filter
Multi-Scale Self-Attention for Text Classification
Qipeng Guo, Xipeng Qiu, Pengfei Liu +2
In this paper, we introduce the prior knowledge, multi-scale structure, into self-attention modules. We propose a Multi-Scale Transformer which uses multi-scale multi-head self-att…
Learning Sparse Sharing Architectures for Multiple Tasks
Tianxiang Sun, Yunfan Shao, Xiaonan Li +4
Most existing deep multi-task learning models are based on parameter sharing, such as hard sharing, hierarchical sharing, and soft sharing. How choosing a suitable sharing mechanis…
A Closer Look at Data Bias in Neural Extractive Summarization Models
Ming Zhong, Danqing Wang, Pengfei Liu +2
In this paper, we take stock of the current state of summarization datasets and explore how different factors of datasets influence the generalization behaviour of neural extractiv…
Exploring Domain Shift in Extractive Text Summarization
Danqing Wang, Pengfei Liu, Ming Zhong +3
Although domain shift has been well explored in many NLP applications, it still has received little attention in the domain of extractive text summarization. As a result, the model…
Zero-shot Text-to-SQL Learning with Auxiliary Task
Shuaichen Chang, Pengfei Liu, Yun Tang +3
Recent years have seen great success in the use of neural seq2seq models on the text-to-SQL task. However, little work has paid attention to how these models generalize to realisti…
DropAttention: A Regularization Method for Fully-Connected Self-Attention Networks
Lin Zehui, Pengfei Liu, Luyao Huang +3
Variants dropout methods have been designed for the fully-connected layer, convolutional layer and recurrent layer in neural networks, and shown to be effective to avoid overfittin…
Searching for Effective Neural Extractive Summarization: What Works and What's Next
Ming Zhong, Pengfei Liu, Danqing Wang +2
The recent years have seen remarkable success in the use of deep neural networks on text summarization. However, there is no clear understanding of \textit{why} they perform so wel…
TIGS: An Inference Algorithm for Text Infilling with Gradient Search
Dayiheng Liu, Jie Fu, Pengfei Liu +1
Text infilling is defined as a task for filling in the missing part of a sentence or paragraph, which is suitable for many real-world natural language generation scenarios. However…
Star-Transformer
Qipeng Guo, Xipeng Qiu, Pengfei Liu +3
Although Transformer has achieved great successes on many NLP tasks, its heavy structure with fully-connected attention connections leads to dependencies on large training data. In…