Lost in Space Marking

Cassandra L. Jacobs, Yuval Pinter

We look at a decision taken early in training a subword tokenizer, namely whether it should be the word-initial token that carries a special mark, or the word-final one. Based on s…