From the 1 of 4 linked papers with an AI index.
1 paper · 1 filter
Violeta Kastreva, Philip Whittington, Dennis Komm +1
Recent works have shown that tokenisation is NP-complete. However, these works assume tokenisation is applied to inputs with unboundedly large alphabets -- an unrealistic assumptio…