The future is different: Large pre-trained language models fail in prediction tasks

Kostadin Cvejoski, Ramsés J. Sánchez, César Ojeda

Large pre-trained language models (LPLM) have shown spectacular success when fine-tuned on downstream supervised tasks. Yet, it is known that their performance can drastically drop…