3 papers
cs.CL2024★ 2 cited
Large Language Models As MOOCs Graders
Shahriar Golchin, Nikhil Garuda, Christopher Impey +1
Massive open online courses (MOOCs) unlock the doors to free education for anyone around the globe with access to a computer and the internet. Despite this democratization of learn…
cs.CL2023
Do not Mask Randomly: Effective Domain-adaptive Pre-training by Masking In-domain Keywords
Shahriar Golchin, Mihai Surdeanu, Nazgol Tavabi +1
We propose a novel task-agnostic in-domain pre-training method that sits between generic pre-training and fine-tuning. Our approach selectively masks in-domain keywords, i.e., word…
cs.CL2022
A Compact Pretraining Approach for Neural Language Models
Shahriar Golchin, Mihai Surdeanu, Nazgol Tavabi +1
Domain adaptation for large neural language models (NLMs) is coupled with massive amounts of unstructured data in the pretraining phase. In this study, however, we show that pretra…