Improving Temporal Generalization of Pre-trained Language Models with Lexical Semantic Change

Zhaochen Su, Zecheng Tang, Xinyan Guan +3

Recent research has revealed that neural language models at scale suffer from poor temporal generalization capability, i.e., the language model pre-trained on static data from past…