Theoretical Analysis of Weak-to-Strong Generalization

Hunter Lang, David Sontag, Aravindan Vijayaraghavan

Strong student models can learn from weaker teachers: when trained on the predictions of a weaker model, a strong pretrained student can learn to correct the weak model's errors an…