Adapting Pretrained Language Models for Solving Tabular Prediction Problems in the Electronic Health Record

Christopher McMaster, David FL Liew, Douglas EV Pires

We propose an approach for adapting the DeBERTa model for electronic health record (EHR) tasks using domain adaptation. We pretrain a small DeBERTa model on a dataset consisting of…