Effects of sub-word segmentation on performance of transformer language models

Jue Hou, Anisia Katinskaia, Anh-Duc Vu +1

Language modeling is a fundamental task in natural language processing, which has been thoroughly explored with various architectures and hyperparameters. However, few studies focu…