Improved Scaling Laws via Weak-to-Strong Generalization in Random Feature Ridge Regression

Diyuan Wu, Lehan Chen, Theodor Misiakiewicz +1

It is increasingly common in machine learning to use learned models to label data and then employ such data to train more capable models. The phenomenon of weak-to-strong generaliz…