718 citations · 1k across the 44 of their papers we have counts for
4 papers · 1 filter
Multi-Scale Self-Attention for Text Classification
Qipeng Guo, Xipeng Qiu, Pengfei Liu +2
In this paper, we introduce the prior knowledge, multi-scale structure, into self-attention modules. We propose a Multi-Scale Transformer which uses multi-scale multi-head self-att…
BP-Transformer: Modelling Long-Range Context via Binary Partitioning
Zihao Ye, Qipeng Guo, Quan Gan +2
The Transformer model is widely successful on many natural language processing tasks. However, the quadratic complexity of self-attention limit its application on long text. In thi…
Deep Graph Library: A Graph-Centric, Highly-Performant Package for Graph Neural Networks
Minjie Wang, Da Zheng, Zihao Ye +12
Advancing research in the emerging field of deep graph learning requires new tools to support tensor computation over graphs. In this paper, we present the design principles and im…
Star-Transformer
Qipeng Guo, Xipeng Qiu, Pengfei Liu +3
Although Transformer has achieved great successes on many NLP tasks, its heavy structure with fully-connected attention connections leads to dependencies on large training data. In…