288 citations · 508 across the 7 of their papers we have counts for
Showing 2021Show all
2 papers · 1 filter
cs.CV2021★ 288 cited
VLMo: Unified Vision-Language Pre-Training with Mixture-of-Modality-Experts
Hangbo Bao, Wenhui Wang, Li Dong +5
We present a unified Vision-Language pretrained Model (VLMo) that jointly learns a dual encoder and a fusion encoder with a modular Transformer network. Specifically, we introduce…
cs.CL2021
Improving Sequence-to-Sequence Pre-training via Sequence Span Rewriting
Wangchunshu Zhou, Tao Ge, Canwen Xu +2
In this paper, we generalize text infilling (e.g., masked language models) by proposing Sequence Span Rewriting (SSR) as a self-supervised sequence-to-sequence (seq2seq) pre-traini…