How Useful is Continued Pre-Training for Generative Unsupervised Domain Adaptation?

Rheeya Uppaal, Yixuan Li, Junjie Hu

Recent breakthroughs in scale have enabled the emergence of powerful generative language models, and the ability to fine-tune these models on various tasks by casting them into pro…