Learning from Synthetic Data: Limitations of ERM

Kareem Amin, Alex Bie, Weiwei Kong +2

The prevalence and low cost of LLMs have led to a rise of synthetic content. From review sites to court documents, "natural" content has been contaminated by data points that appea…