R-grams: Unsupervised Learning of Semantic Units in Natural Language

Ariel Ekgren, Amaru Cuba Gyllensten, Magnus Sahlgren

This paper investigates data-driven segmentation using Re-Pair or Byte Pair Encoding-techniques. In contrast to previous work which has primarily been focused on subword units for…