Showing 2024Show all
2 papers · 1 filter
cs.CL2024
Brevity is the soul of wit: Pruning long files for code generation
Aaditya K. Singh, Yu Yang, Kushal Tirumala +2
Data curation is commonly considered a "secret-sauce" for LLM training, with higher quality data usually leading to better LLM performance. Given the scale of internet-scraped corp…
cs.CV2024
Effective pruning of web-scale datasets based on complexity of concept clusters
Amro Abbas, Evgenia Rusak, Kushal Tirumala +3
Utilizing massive web-scale datasets has led to unprecedented performance gains in machine learning models, but also imposes outlandish compute requirements for their training. In…