416 citations · 3k across the 127 of their papers we have counts for
29 papers · 1 filter
BANG: Bridging Autoregressive and Non-autoregressive Generation with Large Scale Pretraining
Weizhen Qi, Yeyun Gong, Jian Jiao +9
In this paper, we propose BANG, a new pretraining model to Bridge the gap between Autoregressive (AR) and Non-autoregressive (NAR) Generation. AR and NAR generation can be uniforml…
Syntax-Enhanced Pre-trained Model
Zenan Xu, Daya Guo, Duyu Tang +7
We study the problem of leveraging the syntactic structure of text to enhance pre-trained models such as BERT and RoBERTa. Existing methods utilize syntax of text either in the pre…
An Enhanced Knowledge Injection Model for Commonsense Generation
Zhihao Fan, Yeyun Gong, Zhongyu Wei +6
Commonsense generation aims at generating plausible everyday scenario description based on a set of provided concepts. Digging the relationship of concepts from scratch is non-triv…
GLGE: A New General Language Generation Evaluation Benchmark
Dayiheng Liu, Yu Yan, Yeyun Gong +15
Multi-task benchmarks such as GLUE and SuperGLUE have driven great progress of pretraining and transfer learning in Natural Language Processing (NLP). These benchmarks mostly focus…
ProphetNet-Ads: A Looking Ahead Strategy for Generative Retrieval Models in Sponsored Search Engine
Weizhen Qi, Yeyun Gong, Yu Yan +6
In a sponsored search engine, generative retrieval models are recently proposed to mine relevant advertisement keywords for users' input queries. Generative retrieval models genera…
Neural Deepfake Detection with Factual Structure of Text
Wanjun Zhong, Duyu Tang, Zenan Xu +5
Deepfake detection, the task of automatically discriminating machine-generated text, is increasingly critical with recent advances in natural language generative models. Existing a…
Tell Me How to Ask Again: Question Data Augmentation with Controllable Rewriting in Continuous Space
Dayiheng Liu, Yeyun Gong, Jie Fu +5
In this paper, we propose a novel data augmentation method, referred to as Controllable Rewriting based Question Data Augmentation (CRQDA), for machine reading comprehension (MRC),…
No Answer is Better Than Wrong Answer: A Reflection Model for Document Level Machine Reading Comprehension
Xuguang Wang, Linjun Shou, Ming Gong +2
The Natural Questions (NQ) benchmark set brings new challenges to Machine Reading Comprehension: the answers are not only at different levels of granularity (long and short), but a…
GRACE: Gradient Harmonized and Cascaded Labeling for Aspect-based Sentiment Analysis
Huaishao Luo, Lei Ji, Tianrui Li +2
In this paper, we focus on the imbalance issue, which is rarely studied in aspect term extraction and aspect sentiment classification when regarding them as sequence labeling tasks…
Tag and Correct: Question aware Open Information Extraction with Two-stage Decoding
Martin Kuo, Yaobo Liang, Lei Ji +4
Question Aware Open Information Extraction (Question aware Open IE) takes question and passage as inputs, outputting an answer tuple which contains a subject, a predicate, and one…
GraphCodeBERT: Pre-training Code Representations with Data Flow
Daya Guo, Shuo Ren, Shuai Lu +15
Pre-trained models for programming language have achieved dramatic empirical improvements on a variety of code-related tasks such as code search, code completion, code summarizatio…
Evidence-Aware Inferential Text Generation with Vector Quantised Variational AutoEncoder
Daya Guo, Duyu Tang, Nan Duan +3
Generating inferential texts about an event in different perspectives requires reasoning over different contexts that the event occurs. Existing works usually ignore the context th…
M3P: Learning Universal Representations via Multitask Multilingual Multimodal Pre-training
Minheng Ni, Haoyang Huang, Lin Su +6
We present M3P, a Multitask Multilingual Multimodal Pre-trained model that combines multilingual pre-training and multimodal pre-training into a unified framework via multitask pre…
XGLUE: A New Benchmark Dataset for Cross-lingual Pre-training, Understanding and Generation
Yaobo Liang, Nan Duan, Yeyun Gong +21
In this paper, we introduce XGLUE, a new benchmark dataset that can be used to train large-scale cross-lingual pre-trained models using multilingual and bilingual corpora and evalu…
Document Modeling with Graph Attention Networks for Multi-grained Machine Reading Comprehension
Bo Zheng, Haoyang Wen, Yaobo Liang +5
Natural Questions is a new challenging machine reading comprehension benchmark with two-grained answers, which are a long answer (typically a paragraph) and a short answer (one or…
Enhancing Answer Boundary Detection for Multilingual Machine Reading Comprehension
Fei Yuan, Linjun Shou, Xuanyu Bai +5
Multilingual pre-trained models could leverage the training data from a rich source language (such as English) to improve performance on low resource languages. However, the transf…
A Benchmark for Structured Procedural Knowledge Extraction from Cooking Videos
Frank F. Xu, Lei Ji, Botian Shi +4
Watching instructional videos are often used to learn about procedures. Video captioning is one way of automatically collecting such knowledge. However, it provides only an indirec…
RikiNet: Reading Wikipedia Pages for Natural Question Answering
Dayiheng Liu, Yeyun Gong, Jie Fu +5
Reading long documents to answer open-domain questions remains challenging in natural language understanding. In this paper, we introduce a new model, called RikiNet, which reads W…
LogicalFactChecker: Leveraging Logical Operations for Fact Checking with Graph Module Network
Wanjun Zhong, Duyu Tang, Zhangyin Feng +7
Verifying the correctness of a textual statement requires not only semantic reasoning about the meaning of words, but also symbolic reasoning about logical operations like count, s…
A Heterogeneous Graph with Factual, Temporal and Logical Knowledge for Question Answering Over Dynamic Contexts
Wanjun Zhong, Duyu Tang, Nan Duan +3
We study question answering over a dynamic textual environment. Although neural network models achieve impressive accuracy via learning from input-output examples, they rarely leve…
Inferential Text Generation with Multiple Knowledge Sources and Meta-Learning
Daya Guo, Akari Asai, Duyu Tang +6
We study the problem of generating inferential texts of events for a variety of commonsense like \textit{if-else} relations. Existing approaches typically use limited evidence from…
Pre-training Text Representations as Meta Learning
Shangwen Lv, Yuechen Wang, Daya Guo +10
Pre-training text representations has recently been shown to significantly improve the state-of-the-art in many natural language processing tasks. The central goal of pre-training…
Leveraging Declarative Knowledge in Text and First-Order Logic for Fine-Grained Propaganda Detection
Ruize Wang, Duyu Tang, Nan Duan +5
We study the detection of propagandistic text fragments in news articles. Instead of merely learning from input-output datapoints in training data, we introduce an approach to inje…
Diverse, Controllable, and Keyphrase-Aware: A Corpus and Method for News Multi-Headline Generation
Dayiheng Liu, Yeyun Gong, Jie Fu +6
News headline generation aims to produce a short sentence to attract readers to read the news. One news article often contains multiple keyphrases that are of interest to different…