9 citations · 12 across the 3 of their papers we have counts for
3 papers · 1 filter
Evaluation data contamination in LLMs: how do we measure it and (when) does it matter?
Aaditya K. Singh, Muhammed Yusuf Kocyigit, Andrew Poulton +4
Hampering the interpretation of benchmark scores, evaluation data contamination has become a growing concern in the evaluation of LLMs, and an active area of research studies its e…
Brevity is the soul of wit: Pruning long files for code generation
Aaditya K. Singh, Yu Yang, Kushal Tirumala +2
Data curation is commonly considered a "secret-sauce" for LLM training, with higher quality data usually leading to better LLM performance. Given the scale of internet-scraped corp…
Tokenization counts: the impact of tokenization on arithmetic in frontier LLMs
Aaditya K. Singh, DJ Strouse
Tokenization, the division of input text into input tokens, is an often overlooked aspect of the large language model (LLM) pipeline and could be the source of useful or harmful in…