113 citations · 151 across the 3 of their papers we have counts for
3 papers
Recipes for Sequential Pre-training of Multilingual Encoder and Seq2Seq Models
Saleh Soltan, Andy Rosenbaum, Tobias Falke +3
Pre-trained encoder-only and sequence-to-sequence (seq2seq) models each have advantages, however training both model types from scratch is computationally expensive. We explore rec…
AlexaTM 20B: Few-Shot Learning Using a Large-Scale Multilingual Seq2Seq Model
Saleh Soltan, Shankar Ananthakrishnan, Jack FitzGerald +13
In this work, we demonstrate that multilingual large-scale sequence-to-sequence (seq2seq) models, pre-trained on a mixture of denoising and Causal Language Modeling (CLM) tasks, ar…
Multi-Perspective Context Matching for Machine Comprehension
Zhiguo Wang, Haitao Mi, Wael Hamza +1
Previous machine comprehension (MC) datasets are either too small to train end-to-end deep learning models, or not difficult enough to evaluate the ability of current MC techniques…