Back-Translated Task Adaptive Pretraining: Improving Accuracy and Robustness on Text Classification

Junghoon Lee, Jounghee Kim, Pilsung Kang

Language models (LMs) pretrained on a large text corpus and fine-tuned on a downstream text corpus and fine-tuned on a downstream task becomes a de facto training strategy for seve…