Showing cs.CLShow all
2 papers · 1 filter
cs.CL2026
The Lexical Imprint of Pre-training Data
Woojin Chung, Jeonghoon Kim, Woomin Song +3
How much of zero-shot benchmark accuracy is captured by word-level alignment between a pre-training corpus and the benchmark target? We study this question through word-level unigr…
cs.CL2025
Exploiting Vocabulary Frequency Imbalance in Language Model Pre-training
Woojin Chung, Jeonghoon Kim
Large language models are trained with tokenizers, and the resulting token distribution is highly imbalanced: a few words dominate the stream while most occur rarely. Recent practi…