Towards Simple and Efficient Task-Adaptive Pre-training for Text Classification

Arnav Ladkat, Aamir Miyajiwala, Samiksha Jagadale +2

Language models are pre-trained using large corpora of generic data like book corpus, common crawl and Wikipedia, which is essential for the model to understand the linguistic char…