254 citations · 312 across the 10 of their papers we have counts for
1 paper · 1 filter
Aaditya K. Singh, DJ Strouse
Tokenization, the division of input text into input tokens, is an often overlooked aspect of the large language model (LLM) pipeline and could be the source of useful or harmful in…