Language Model Adaptation to Specialized Domains through Selective Masking based on Genre and Topical Characteristics

Anas Belfathi, Ygor Gallina, Nicolas Hernandez +2

Recent advances in pre-trained language modeling have facilitated significant progress across various natural language processing (NLP) tasks. Word masking during model training co…