Does Pretraining Teach Chemical Language Models Chemistry?

Leif Sieben, Yoel Zimmermann

Chemical language models (CLMs) are pretrained on large corpora of molecules with the aim of learning representations that transfer to diverse downstream tasks. Here, we systematic…