50 citations · 51 across the 2 of their papers we have counts for
2 papers
cs.CL2024★ 1 cited
Rethinking Tokenization: Crafting Better Tokenizers for Large Language Models
Jinbiao Yang
Tokenization significantly influences language models(LMs)' performance. This paper traces the evolution of tokenizers from word-level to subword-level, analyzing how they balance…
q-bio.NC2017★ 50 cited
Lexical representation explains cortical entrainment during speech comprehension
Stefan Frank, Jinbiao Yang
Results from a recent neuroimaging study on spoken sentence comprehension have been interpreted as evidence for cortical entrainment to hierarchical syntactic structure. We present…