416 citations · 1.6k across the 85 of their papers we have counts for
25 papers · 2 filters
Reinforced Multi-Teacher Selection for Knowledge Distillation
Fei Yuan, Linjun Shou, Jian Pei +4
In natural language processing (NLP) tasks, slow inference speed and huge footprints in GPU usage remain the bottleneck of applying pre-trained deep models in production. As a popu…
Syntax-Enhanced Pre-trained Model
Zenan Xu, Daya Guo, Duyu Tang +7
We study the problem of leveraging the syntactic structure of text to enhance pre-trained models such as BERT and RoBERTa. Existing methods utilize syntax of text either in the pre…
CalibreNet: Calibration Networks for Multilingual Sequence Labeling
Shining Liang, Linjun Shou, Jian Pei +3
Lack of training data in low-resource languages presents huge challenges to sequence labeling tasks such as named entity recognition (NER) and machine reading comprehension (MRC).…
GLGE: A New General Language Generation Evaluation Benchmark
Dayiheng Liu, Yu Yan, Yeyun Gong +15
Multi-task benchmarks such as GLUE and SuperGLUE have driven great progress of pretraining and transfer learning in Natural Language Processing (NLP). These benchmarks mostly focus…
Cross-lingual Machine Reading Comprehension with Language Branch Knowledge Distillation
Junhao Liu, Linjun Shou, Jian Pei +3
Cross-lingual Machine Reading Comprehension (CLMRC) remains a challenging problem due to the lack of large-scale annotated datasets in low-source languages, such as Arabic, Hindi,…
A Graph Representation of Semi-structured Data for Web Question Answering
Xingyao Zhang, Linjun Shou, Jian Pei +3
The abundant semi-structured data on the Web, such as HTML-based tables and lists, provide commercial search engines a rich information source for question answering (QA). Differen…
Towards Interpretable Reasoning over Paragraph Effects in Situation
Mucheng Ren, Xiubo Geng, Tao Qin +2
We focus on the task of reasoning over paragraph effects in situation, which requires a model to understand the cause and effect described in a background paragraph, and apply the…
No Answer is Better Than Wrong Answer: A Reflection Model for Document Level Machine Reading Comprehension
Xuguang Wang, Linjun Shou, Ming Gong +2
The Natural Questions (NQ) benchmark set brings new challenges to Machine Reading Comprehension: the answers are not only at different levels of granularity (long and short), but a…
GRACE: Gradient Harmonized and Cascaded Labeling for Aspect-based Sentiment Analysis
Huaishao Luo, Lei Ji, Tianrui Li +2
In this paper, we focus on the imbalance issue, which is rarely studied in aspect term extraction and aspect sentiment classification when regarding them as sequence labeling tasks…
Difference-aware Knowledge Selection for Knowledge-grounded Conversation Generation
Chujie Zheng, Yunbo Cao, Daxin Jiang +1
In a multi-turn knowledge-grounded dialog, the difference between the knowledge selected at different turns usually provides potential clues to knowledge selection, which has been…
Learning an Effective Context-Response Matching Model with Self-Supervised Tasks for Retrieval-based Dialogues
Ruijian Xu, Chongyang Tao, Daxin Jiang +3
Building an intelligent dialogue system with the ability to select a proper response according to a multi-turn context is a great challenging task. Existing studies focus on buildi…
Knowledge-Aware Procedural Text Understanding with Multi-Stage Training
Zhihan Zhang, Xiubo Geng, Tao Qin +2
Procedural text describes dynamic state changes during a step-by-step natural process (e.g., photosynthesis). In this work, we focus on the task of procedural text understanding, w…
Evidence-Aware Inferential Text Generation with Vector Quantised Variational AutoEncoder
Daya Guo, Duyu Tang, Nan Duan +3
Generating inferential texts about an event in different perspectives requires reasoning over different contexts that the event occurs. Existing works usually ignore the context th…
XGLUE: A New Benchmark Dataset for Cross-lingual Pre-training, Understanding and Generation
Yaobo Liang, Nan Duan, Yeyun Gong +21
In this paper, we introduce XGLUE, a new benchmark dataset that can be used to train large-scale cross-lingual pre-trained models using multilingual and bilingual corpora and evalu…
Document Modeling with Graph Attention Networks for Multi-grained Machine Reading Comprehension
Bo Zheng, Haoyang Wen, Yaobo Liang +5
Natural Questions is a new challenging machine reading comprehension benchmark with two-grained answers, which are a long answer (typically a paragraph) and a short answer (one or…
Enhancing Answer Boundary Detection for Multilingual Machine Reading Comprehension
Fei Yuan, Linjun Shou, Xuanyu Bai +5
Multilingual pre-trained models could leverage the training data from a rich source language (such as English) to improve performance on low resource languages. However, the transf…
RikiNet: Reading Wikipedia Pages for Natural Question Answering
Dayiheng Liu, Yeyun Gong, Jie Fu +5
Reading long documents to answer open-domain questions remains challenging in natural language understanding. In this paper, we introduce a new model, called RikiNet, which reads W…
LogicalFactChecker: Leveraging Logical Operations for Fact Checking with Graph Module Network
Wanjun Zhong, Duyu Tang, Zhangyin Feng +7
Verifying the correctness of a textual statement requires not only semantic reasoning about the meaning of words, but also symbolic reasoning about logical operations like count, s…
Inferential Text Generation with Multiple Knowledge Sources and Meta-Learning
Daya Guo, Akari Asai, Duyu Tang +6
We study the problem of generating inferential texts of events for a variety of commonsense like \textit{if-else} relations. Existing approaches typically use limited evidence from…
Pre-training Text Representations as Meta Learning
Shangwen Lv, Yuechen Wang, Daya Guo +10
Pre-training text representations has recently been shown to significantly improve the state-of-the-art in many natural language processing tasks. The central goal of pre-training…
Leveraging Declarative Knowledge in Text and First-Order Logic for Fine-Grained Propaganda Detection
Ruize Wang, Duyu Tang, Nan Duan +5
We study the detection of propagandistic text fragments in news articles. Instead of merely learning from input-output datapoints in training data, we introduce an approach to inje…
Diverse, Controllable, and Keyphrase-Aware: A Corpus and Method for News Multi-Headline Generation
Dayiheng Liu, Yeyun Gong, Jie Fu +6
News headline generation aims to produce a short sentence to attract readers to read the news. One news article often contains multiple keyphrases that are of interest to different…
DC-BERT: Decoupling Question and Document for Efficient Contextual Encoding
Yuyu Zhang, Ping Nie, Xiubo Geng +3
Recent studies on open-domain question answering have achieved prominent performance improvement using pre-trained language models such as BERT. State-of-the-art approaches typical…
CodeBERT: A Pre-Trained Model for Programming and Natural Languages
Zhangyin Feng, Daya Guo, Duyu Tang +8
We present CodeBERT, a bimodal pre-trained model for programming language (PL) and nat-ural language (NL). CodeBERT learns general-purpose representations that support downstream N…