437 citations · 1.4k across the 31 of their papers we have counts for
10 papers · 1 filter
A Tale of Two Linkings: Dynamically Gating between Schema Linking and Structural Linking for Text-to-SQL Parsing
Sanxing Chen, Aidan San, Xiaodong Liu +1
In Text-to-SQL semantic parsing, selecting the correct entities (tables and columns) for the generated SQL query is both crucial and challenging; the parser is required to connect…
Generation-Augmented Retrieval for Open-domain Question Answering
Yuning Mao, Pengcheng He, Xiaodong Liu +4
We propose Generation-Augmented Retrieval (GAR) for answering open-domain questions, which augments a query through text generation of heuristically discovered relevant contexts wi…
HittER: Hierarchical Transformers for Knowledge Graph Embeddings
Sanxing Chen, Xiaodong Liu, Jianfeng Gao +3
This paper examines the challenging problem of learning representations of entities and relations in a complex multi-relational knowledge graph. We propose HittER, a Hierarchical T…
Very Deep Transformers for Neural Machine Translation
Xiaodong Liu, Kevin Duh, Liyuan Liu +1
We explore the application of very deep Transformer models for Neural Machine Translation (NMT). Using a simple yet effective initialization technique that stabilizes training, we…
DeBERTa: Decoding-enhanced BERT with Disentangled Attention
Pengcheng He, Xiaodong Liu, Jianfeng Gao +1
Recent progress in pre-trained neural language models has significantly improved the performance of many natural language processing (NLP) tasks. In this paper we propose a new mod…
Adversarial Training for Commonsense Inference
Lis Pereira, Xiaodong Liu, Fei Cheng +2
We propose an AdversariaL training algorithm for commonsense InferenCE (ALICE). We apply small perturbations to word embeddings and minimize the resultant adversarial risk to regul…
Adversarial Training for Large Neural Language Models
Xiaodong Liu, Hao Cheng, Pengcheng He +4
Generalization and robustness are both key desiderata for designing machine learning methods. Adversarial training can enhance robustness, but past work often finds it hurts genera…
Understanding the Difficulty of Training Transformers
Liyuan Liu, Xiaodong Liu, Jianfeng Gao +2
Transformers have proved effective in many NLP tasks. However, their training requires non-trivial efforts regarding designing cutting-edge optimizers and learning rate schedulers…
UniLMv2: Pseudo-Masked Language Models for Unified Language Model Pre-Training
Hangbo Bao, Li Dong, Furu Wei +8
We propose to pre-train a unified language model for both autoencoding and partially autoregressive language modeling tasks using a novel training procedure, referred to as a pseud…
The Microsoft Toolkit of Multi-Task Deep Neural Networks for Natural Language Understanding
Xiaodong Liu, Yu Wang, Jianshu Ji +8
We present MT-DNN, an open-source natural language understanding (NLU) toolkit that makes it easy for researchers and developers to train customized deep learning models. Built upo…