Brevity is the soul of wit: Pruning long files for code generation

Aaditya K. Singh, Yu Yang, Kushal Tirumala +2

Data curation is commonly considered a "secret-sauce" for LLM training, with higher quality data usually leading to better LLM performance. Given the scale of internet-scraped corp…