Explicit Morphological Knowledge Improves Pre-training of Language Models for Hebrew

Eylon Gueta, Omer Goldman, Reut Tsarfaty

Pre-trained language models (PLMs) have shown remarkable successes in acquiring a wide range of linguistic knowledge, relying solely on self-supervised training on text streams. Ne…