Do Language Models Learn about Legal Entity Types during Pretraining?

Claire Barale, Michael Rovatsos, Nehal Bhuta

Language Models (LMs) have proven their ability to acquire diverse linguistic knowledge during the pretraining phase, potentially serving as a valuable source of incidental supervi…