12 citations · 14 across the 7 of their papers we have counts for
Showing 2026Show all
2 papers · 1 filter
cs.LG2026
Rewriting Scaling Laws for Synthetic Data with Compute and Data Constraints
Nicholas Roberts, Ahmed Ahmed, Suhas Kotha +4
Synthetic rewrites, where a language model rewrites a natural document e.g. into a target style, are now a standard ingredient of pretraining corpora. However, critical aspects of…
cs.LG2026
Data-efficient pre-training by scaling synthetic megadocs
Konwoo Kim, Suhas Kotha, Yejin Choi +3
Synthetic data augmentation has emerged as a promising solution when pre-training is constrained by data rather than compute. We study how to design synthetic data algorithms that…