9 citations · 11 across the 14 of their papers we have counts for
Showing cs.LGShow all
2 papers · 1 filter
cs.LG2026
An Empirical Study on Noisy Data and LLM Pretraining Loss Divergence
Qizhen Zhang, Ankush Garg, Jakob Foerster +3
Large-scale pretraining datasets drive the success of large language models (LLMs). However, these web-scale corpora inevitably contain large amounts of noisy data due to unregulat…
cs.LG2025
Compute Optimal Scaling of Skills: Knowledge vs Reasoning
Nicholas Roberts, Niladri Chatterji, Sharan Narang +2
Scaling laws are a critical component of the LLM development pipeline, most famously as a way to forecast training decisions such as 'compute-optimally' trading-off parameter count…