Efficient Hierarchical Domain Adaptation for Pretrained Language Models

Alexandra Chronopoulou, Matthew E. Peters, Jesse Dodge

The remarkable success of large language models has been driven by dense models trained on massive unlabeled, unstructured corpora. These corpora typically contain text from divers…