1.9k citations · 10.4k across the 198 of their papers we have counts for
13 papers · 1 filter
LayoutLM: Pre-training of Text and Layout for Document Image Understanding
Yiheng Xu, Minghao Li, Lei Cui +3
Pre-training techniques have been verified successfully in a variety of NLP tasks in recent years. Despite the widespread use of pre-training models for NLP applications, they almo…
Can Monolingual Pretrained Models Help Cross-Lingual Classification?
Zewen Chi, Li Dong, Furu Wei +2
Multilingual pretrained language models (such as multilingual BERT) have achieved impressive results for cross-lingual transfer. However, due to the constant model capacity, multil…
Transforming Wikipedia into Augmented Data for Query-Focused Summarization
Haichao Zhu, Li Dong, Furu Wei +2
The limited size of existing query-focused summarization datasets renders training data-driven summarization models challenging. Meanwhile, the manual construction of a query-focus…
Improving Grammatical Error Correction with Machine Translation Pairs
Wangchunshu Zhou, Tao Ge, Chang Mu +3
We propose a novel data synthesis method to generate diverse error-corrected sentence pairs for improving grammatical error correction, which is based on a pair of machine translat…
Sequence-to-sequence Pre-training with Data Augmentation for Sentence Rewriting
Yi Zhang, Tao Ge, Furu Wei +2
We study sequence-to-sequence (seq2seq) pre-training with data augmentation for sentence rewriting. Instead of training a seq2seq model with gold training data and augmented data s…
Cross-Lingual Natural Language Generation via Pre-Training
Zewen Chi, Li Dong, Furu Wei +3
In this work we focus on transferring supervision signals of natural language generation (NLG) tasks between multiple languages. We propose to pretrain the encoder and the decoder…
Visualizing and Understanding the Effectiveness of BERT
Yaru Hao, Li Dong, Furu Wei +1
Language model pre-training, such as BERT, has achieved remarkable results in many NLP tasks. However, it is unclear why the pre-training-then-fine-tuning paradigm can improve perf…
VL-BERT: Pre-training of Generic Visual-Linguistic Representations
Weijie Su, Xizhou Zhu, Yue Cao +4
We introduce a new pre-trainable generic representation for visual-linguistic tasks, called Visual-Linguistic BERT (VL-BERT for short). VL-BERT adopts the simple yet powerful Trans…
Learning to Ask Unanswerable Questions for Machine Reading Comprehension
Haichao Zhu, Li Dong, Furu Wei +3
Machine reading comprehension with unanswerable questions is a challenging task. In this work, we propose a data augmentation technique by automatically generating relevant unanswe…
HIBERT: Document Level Pre-training of Hierarchical Bidirectional Transformers for Document Summarization
Xingxing Zhang, Furu Wei, Ming Zhou
Neural extractive summarization models usually employ a hierarchical encoder for document encoding and they are trained using sentence-level labels, which are created heuristically…
Unified Language Model Pre-training for Natural Language Understanding and Generation
Li Dong, Nan Yang, Wenhui Wang +6
This paper presents a new Unified pre-trained Language Model (UniLM) that can be fine-tuned for both natural language understanding and generation tasks. The model is pre-trained u…
Formality Style Transfer with Hybrid Textual Annotations
Ruochen Xu, Tao Ge, Furu Wei
Formality style transformation is the task of modifying the formality of a given sentence without changing its content. Its challenge is the lack of large-scale sentence-aligned pa…
TableBank: A Benchmark Dataset for Table Detection and Recognition
Minghao Li, Lei Cui, Shaohan Huang +3
We present TableBank, a new image-based table detection and recognition dataset built with novel weak supervision from Word and Latex documents on the internet. Existing research f…