40 citations · 81 across the 6 of their papers we have counts for
4 papers · 1 filter
Contrastive Document Representation Learning with Graph Attention Networks
Peng Xu, Xinchi Chen, Xiaofei Ma +2
Recent progress in pretrained Transformer-based language models has shown great success in learning contextual representation of text. However, due to the quadratic self-attention…
Beyond [CLS] through Ranking by Generation
Cicero Nogueira dos Santos, Xiaofei Ma, Ramesh Nallapati +2
Generative models for Information Retrieval, where ranking of documents is viewed as the task of generating a query from a document's language model, were very successful in variou…
Universal Text Representation from BERT: An Empirical Study
Xiaofei Ma, Zhiguo Wang, Patrick Ng +2
We present a systematic investigation of layer-wise BERT activations for general-purpose text representations to understand what linguistic information they capture and how transfe…
Multi-passage BERT: A Globally Normalized BERT Model for Open-domain Question Answering
Zhiguo Wang, Patrick Ng, Xiaofei Ma +2
BERT model has been successfully applied to open-domain QA tasks. However, previous work trains BERT by viewing passages corresponding to the same question as independent training…