WaLDORf: Wasteless Language-model Distillation On Reading-comprehension

James Yi Tian, Alexander P. Kreuzer, Pai-Hung Chen +1

Transformer based Very Large Language Models (VLLMs) like BERT, XLNet and RoBERTa, have recently shown tremendous performance on a large variety of Natural Language Understanding (…