PRG: Prompt-Based Distillation Without Annotation via Proxy Relational Graph

Yijin Xu, Jialun Liu, Hualiang Wei +1

In this paper, we propose a new distillation method for extracting knowledge from Large Foundation Models (LFM) into lightweight models, introducing a novel supervision mode that d…