Are word boundaries useful for unsupervised language learning?

Tu Anh Nguyen, Maureen de Seyssel, Robin Algayres +3

Word or word-fragment based Language Models (LM) are typically preferred over character-based ones in many downstream applications. This may not be surprising as words seem more li…