Showing cs.LGShow all
2 papers · 1 filter
cs.LG2025
ReMem: Mutual Information-Aware Fine-tuning of Pretrained Vision Transformers for Effective Knowledge Distillation
Chengyu Dong, Huan Gui, Noveen Sachdeva +6
Knowledge distillation from pretrained visual representation models offers an effective approach to improve small, task-specific production models. However, the effectiveness of su…
cs.LG2024
LEVI: Generalizable Fine-tuning via Layer-wise Ensemble of Different Views
Yuji Roh, Qingyun Liu, Huan Gui +8
Fine-tuning is becoming widely used for leveraging the power of pre-trained foundation models in new downstream tasks. While there are many successes of fine-tuning on various task…