416 citations · 1k across the 55 of their papers we have counts for
17 papers · 1 filter
Reinforced Multi-Teacher Selection for Knowledge Distillation
Fei Yuan, Linjun Shou, Jian Pei +4
In natural language processing (NLP) tasks, slow inference speed and huge footprints in GPU usage remain the bottleneck of applying pre-trained deep models in production. As a popu…
Syntax-Enhanced Pre-trained Model
Zenan Xu, Daya Guo, Duyu Tang +7
We study the problem of leveraging the syntactic structure of text to enhance pre-trained models such as BERT and RoBERTa. Existing methods utilize syntax of text either in the pre…
CalibreNet: Calibration Networks for Multilingual Sequence Labeling
Shining Liang, Linjun Shou, Jian Pei +3
Lack of training data in low-resource languages presents huge challenges to sequence labeling tasks such as named entity recognition (NER) and machine reading comprehension (MRC).…
GLGE: A New General Language Generation Evaluation Benchmark
Dayiheng Liu, Yu Yan, Yeyun Gong +15
Multi-task benchmarks such as GLUE and SuperGLUE have driven great progress of pretraining and transfer learning in Natural Language Processing (NLP). These benchmarks mostly focus…
Cross-lingual Machine Reading Comprehension with Language Branch Knowledge Distillation
Junhao Liu, Linjun Shou, Jian Pei +3
Cross-lingual Machine Reading Comprehension (CLMRC) remains a challenging problem due to the lack of large-scale annotated datasets in low-source languages, such as Arabic, Hindi,…
A Graph Representation of Semi-structured Data for Web Question Answering
Xingyao Zhang, Linjun Shou, Jian Pei +3
The abundant semi-structured data on the Web, such as HTML-based tables and lists, provide commercial search engines a rich information source for question answering (QA). Differen…
MaP: A Matrix-based Prediction Approach to Improve Span Extraction in Machine Reading Comprehension
Huaishao Luo, Yu Shi, Ming Gong +2
Span extraction is an essential problem in machine reading comprehension. Most of the existing algorithms predict the start and end positions of an answer span in the given corresp…
No Answer is Better Than Wrong Answer: A Reflection Model for Document Level Machine Reading Comprehension
Xuguang Wang, Linjun Shou, Ming Gong +2
The Natural Questions (NQ) benchmark set brings new challenges to Machine Reading Comprehension: the answers are not only at different levels of granularity (long and short), but a…
Tag and Correct: Question aware Open Information Extraction with Two-stage Decoding
Martin Kuo, Yaobo Liang, Lei Ji +4
Question Aware Open Information Extraction (Question aware Open IE) takes question and passage as inputs, outputting an answer tuple which contains a subject, a predicate, and one…
Mining Implicit Relevance Feedback from User Behavior for Web Question Answering
Linjun Shou, Shining Bo, Feixiang Cheng +3
Training and refreshing a web-scale Question Answering (QA) system for a multi-lingual commercial search engine often requires a huge amount of training examples. One principled id…
XGLUE: A New Benchmark Dataset for Cross-lingual Pre-training, Understanding and Generation
Yaobo Liang, Nan Duan, Yeyun Gong +21
In this paper, we introduce XGLUE, a new benchmark dataset that can be used to train large-scale cross-lingual pre-trained models using multilingual and bilingual corpora and evalu…
Enhancing Answer Boundary Detection for Multilingual Machine Reading Comprehension
Fei Yuan, Linjun Shou, Xuanyu Bai +5
Multilingual pre-trained models could leverage the training data from a rich source language (such as English) to improve performance on low resource languages. However, the transf…
LogicalFactChecker: Leveraging Logical Operations for Fact Checking with Graph Module Network
Wanjun Zhong, Duyu Tang, Zhangyin Feng +7
Verifying the correctness of a textual statement requires not only semantic reasoning about the meaning of words, but also symbolic reasoning about logical operations like count, s…
Inferential Text Generation with Multiple Knowledge Sources and Meta-Learning
Daya Guo, Akari Asai, Duyu Tang +6
We study the problem of generating inferential texts of events for a variety of commonsense like \textit{if-else} relations. Existing approaches typically use limited evidence from…
Pre-training Text Representations as Meta Learning
Shangwen Lv, Yuechen Wang, Daya Guo +10
Pre-training text representations has recently been shown to significantly improve the state-of-the-art in many natural language processing tasks. The central goal of pre-training…
Improving Readability for Automatic Speech Recognition Transcription
Junwei Liao, Sefik Emre Eskimez, Liyang Lu +5
Modern Automatic Speech Recognition (ASR) systems can achieve high performance in terms of recognition accuracy. However, a perfectly accurate transcript still can be challenging t…
CodeBERT: A Pre-Trained Model for Programming and Natural Languages
Zhangyin Feng, Daya Guo, Duyu Tang +8
We present CodeBERT, a bimodal pre-trained model for programming language (PL) and nat-ural language (NL). CodeBERT learns general-purpose representations that support downstream N…