687 citations · 4k across the 58 of their papers we have counts for
4 papers · 1 filter
LayoutLM: Pre-training of Text and Layout for Document Image Understanding
Yiheng Xu, Minghao Li, Lei Cui +3
Pre-training techniques have been verified successfully in a variety of NLP tasks in recent years. Despite the widespread use of pre-training models for NLP applications, they almo…
Semantic Mask for Transformer based End-to-End Speech Recognition
Chengyi Wang, Yu Wu, Yujiao Du +7
Attention-based encoder-decoder model has achieved impressive results for both automatic speech recognition (ASR) and text-to-speech (TTS) tasks. This approach takes advantage of t…
Sequence-to-sequence Pre-training with Data Augmentation for Sentence Rewriting
Yi Zhang, Tao Ge, Furu Wei +2
We study sequence-to-sequence (seq2seq) pre-training with data augmentation for sentence rewriting. Instead of training a seq2seq model with gold training data and augmented data s…
Unicoder-VL: A Universal Encoder for Vision and Language by Cross-modal Pre-training
Gen Li, Nan Duan, Yuejian Fang +3
We propose Unicoder-VL, a universal encoder that aims to learn joint representations of vision and language in a pre-training manner. Borrow ideas from cross-lingual pre-trained mo…