Self-Verified Distillation: Your Language Model Is Secretly Its Own Synthetic Data Pipeline

Tony Lee, Percy Liang

Can post-trained large language models (LLMs) further improve themselves using only unlabeled prompts, without external teachers or feedback from tools? We study this setting start…