CLIP-RD: Relational Distillation for Efficient CLIP Knowledge Distillation

Jeannie Chung, Hanna Jang, Ingyeong Yang +2

Contrastive Language-Image Pre-training (CLIP) demonstrates strong zero-shot generalization, but due to substantial computational and memory costs, distillation into lightweight mo…