Learning to Look Inside: Augmenting Token-Based Encoders with Character-Level Information

Yuval Pinter, Amanda Stent, Mark Dredze +1

Commonly-used transformer language models depend on a tokenization schema which sets an unchangeable subword vocabulary prior to pre-training, destined to be applied to all downstr…