SIGMA: Scalable Spectral Insights for LLM Model Collapse

Yi Gu, Lingyou Pang, Xiangkun Ye +4

The rapid adoption of synthetic data for training Large Language Models (LLMs) has introduced the technical challenge of "model collapse"-a degenerative process where recursive tra…