Learning with Synthetic Data via SGD in High-Dimensional Linear Regression

Jichu li, Difan Zou

Synthetic data has become a promising way to scale model training beyond limited human-generated data but it may also induce strong model collapse (Dohmatob et al., 2024), where an…