On the Blessing of Pre-training in Weak-to-Strong Generalization

Wei Yao, Wang Zhaoyang, Gengze Xu +5

The paradigm of Weak-to-Strong Generalization (W2SG) suggests that a pre-trained strong model can surpass its weak supervisor, yet the decisive role of pre-training remains theoret…