9 citations · 9 across the 3 of their papers we have counts for
1 paper · 1 filter
Harshit Verma, Rex Ying
Exact incremental tokenizers keep the tokenization of a growing text up to date, so a long prompt or agent trace is not retokenized each time it grows. Jiang and Gong's algorithm d…