24 citations · 39 across the 5 of their papers we have counts for
1 paper · 1 filter
Martin Berglund, Brink van der Merwe
In this paper, we formalize practical byte pair encoding tokenization as it is used in large language models and other NLP systems, in particular we formally define and investigate…