Multistage Collaborative Knowledge Distillation from a Large Language Model for Semi-Supervised Sequence Generation

Jiachen Zhao, Wenlong Zhao, Andrew Drozdov +5

We study semi-supervised sequence generation tasks, where the few labeled examples are too scarce to finetune a model, and meanwhile, few-shot prompted large language models (LLMs)…