Learning from Saturated Data: Signals Beyond Correctness for LLM Training

Hanno Hiss, Jasper Dekoninck, Martin Vechev

The growing capabilities of large language models (LLMs) have led to the saturation of many benchmarks and training datasets used to improve them. Motivated by this, we investigate…