Distilling Cross-Modal Knowledge via Feature Disentanglement

Junhong Liu, Yuan Zhang, Tao Huang +2

Knowledge distillation (KD) has proven highly effective for compressing large models and enhancing the performance of smaller ones. However, its effectiveness diminishes in cross-m…