652 citations · 1.7k across the 25 of their papers we have counts for
Showing 2021 · cs.CLShow all
3 papers · 2 filters
cs.CL2021
Contrastive Document Representation Learning with Graph Attention Networks
Peng Xu, Xinchi Chen, Xiaofei Ma +2
Recent progress in pretrained Transformer-based language models has shown great success in learning contextual representation of text. However, due to the quadratic self-attention…
cs.CL2021★ 11 cited
Attention-guided Generative Models for Extractive Question Answering
Peng Xu, Davis Liang, Zhiheng Huang +1
We propose a novel method for applying Transformer models to extractive question answering (QA) tasks. Recently, pretrained generative sequence-to-sequence (seq2seq) models have ac…
cs.CL2021★ 1 cited
Multiplicative Position-aware Transformer Models for Language Understanding
Zhiheng Huang, Davis Liang, Peng Xu +1
Transformer models, which leverage architectural improvements like self-attention, perform remarkably well on Natural Language Processing (NLP) tasks. The self-attention mechanism…