Towards understanding evolution of science through language model series

Junjie Dong, Zhuoqi Lyu, Qing Ke

We introduce AnnualBERT, a series of language models designed specifically to capture the temporal evolution of scientific text. Deviating from the prevailing paradigms of subword…