An Active-Bottleneck Mechanism for Weak-to-Strong Generalization

Mohammad Zeinalpour, Amir Najafi

Weak-to-strong generalization (W2SG) occurs when a student trained on a teacher's predictions outperforms that teacher. We study when this happens under fully converged, ridgeless…