Regurgitative Training: The Value of Real Data in Training Large Language Models

Jinghui Zhang, Dandan Qiao, Mochen Yang +1

What happens if we train a new Large Language Model (LLM) using data that are at least partially generated by other LLMs? The explosive success of LLMs means that a substantial amo…