How Long Is Enough? Exploring the Optimal Intervals of Long-Range Clinical Note Language Modeling

Samuel Cahyawijaya, Bryan Wilie, Holy Lovenia +4

Large pre-trained language models (LMs) have been widely adopted in biomedical and clinical domains, introducing many powerful LMs such as bio-lm and BioELECTRA. However, the appli…