most citedDo All Languages Cost the Same? Tokenization in the Era of Commercial Language Models

3 citations · 3 across the 1 of their papers we have counts for

collaborators

1 paper

cs.CL2023★ 3 cited

Do All Languages Cost the Same? Tokenization in the Era of Commercial Language Models

Orevaoghene Ahia, Sachin Kumar, Hila Gonen +4

Language models have graduated from being research prototypes to commercialized products offered as web APIs, and recent works have highlighted the multilingual capabilities of the…