Talking Models: Distill Pre-trained Knowledge to Downstream Models via Interactive Communication

Zhe Zhao, Qingyun Liu, Huan Gui +3

Many recent breakthroughs in machine learning have been enabled by the pre-trained foundation models. By scaling up model parameters, training data, and computation resources, foun…