384 citations · 640 across the 13 of their papers we have counts for
9 papers · 1 filter
StructuralLM: Structural Pre-training for Form Understanding
Chenliang Li, Bin Bi, Ming Yan +4
Large pre-trained language models achieve state-of-the-art results when fine-tuned on downstream NLP tasks. However, they almost exclusively focus on text-only representation, whil…
SemVLP: Vision-Language Pre-training by Aligning Semantics at Multiple Levels
Chenliang Li, Ming Yan, Haiyang Xu +4
Vision-language pre-training (VLP) on large-scale image-text pairs has recently witnessed rapid progress for learning cross-modal representations. Existing pre-training methods eit…
CoRe: An Efficient Coarse-refined Training Framework for BERT
Cheng Yang, Shengnan Wang, Yuechuan Li +4
In recent years, BERT has made significant breakthroughs on many natural language processing tasks and attracted great attentions. Despite its accuracy gains, the BERT model genera…
PALM: Pre-training an Autoencoding&Autoregressive Language Model for Context-conditioned Generation
Bin Bi, Chenliang Li, Chen Wu +5
Self-supervised pre-training, such as BERT, MASS and BART, has emerged as a powerful technique for natural language understanding and generation. Existing pre-training techniques e…
StructBERT: Incorporating Language Structures into Pre-training for Deep Language Understanding
Wei Wang, Bin Bi, Ming Yan +5
Recently, the pre-trained language model, BERT (and its robustly optimized version RoBERTa), has attracted a lot of attention in natural language understanding (NLU), and achieved…
Incorporating External Knowledge into Machine Reading for Generative Question Answering
Bin Bi, Chen Wu, Ming Yan +3
Commonsense and background knowledge is required for a QA model to answer many nontrivial questions. Different from existing work on knowledge-aware QA, we focus on a more challeng…