TinyCLIP: CLIP Distillation via Affinity Mimicking and Weight Inheritance

Kan Wu, Houwen Peng, Zhenghong Zhou +10

In this paper, we propose a novel cross-modal distillation method, called TinyCLIP, for large-scale language-image pre-trained models. The method introduces two core techniques: af…