A Masked Segmental Language Model for Unsupervised Natural Language Segmentation

C. M. Downey, Fei Xia, Gina-Anne Levow +1

Segmentation remains an important preprocessing step both in languages where "words" or other important syntactic/semantic units (like morphemes) are not clearly delineated by whit…