Variational Pretraining for Semi-supervised Text Classification

Suchin Gururangan, Tam Dang, Dallas Card +1

We introduce VAMPIRE, a lightweight pretraining framework for effective text classification when data and computing resources are limited. We pretrain a unigram document model as a…