3 papers
cs.LG2025
ReMem: Mutual Information-Aware Fine-tuning of Pretrained Vision Transformers for Effective Knowledge Distillation
Chengyu Dong, Huan Gui, Noveen Sachdeva +6
Knowledge distillation from pretrained visual representation models offers an effective approach to improve small, task-specific production models. However, the effectiveness of su…
cs.IR2023
Hiformer: Heterogeneous Feature Interactions Learning with Transformers for Recommender Systems
Huan Gui, Ruoxi Wang, Ke Yin +5
Learning feature interaction is the critical backbone to building recommender systems. In web-scale applications, learning feature interaction is extremely challenging due to the s…
cs.AI2023
Talking Models: Distill Pre-trained Knowledge to Downstream Models via Interactive Communication
Zhe Zhao, Qingyun Liu, Huan Gui +3
Many recent breakthroughs in machine learning have been enabled by the pre-trained foundation models. By scaling up model parameters, training data, and computation resources, foun…