2 citations · 3 across the 3 of their papers we have counts for
3 papers
How to Train the Teacher Model for Effective Knowledge Distillation
Shayan Mohajer Hamidi, Xizhen Deng, Renhao Tan +2
Recently, it was shown that the role of the teacher in knowledge distillation (KD) is to provide the student with an estimate of the true Bayes conditional probability density (BCP…
Bayes Conditional Distribution Estimation for Knowledge Distillation Based on Conditional Mutual Information
Linfeng Ye, Shayan Mohajer Hamidi, Renhao Tan +1
It is believed that in knowledge distillation (KD), the role of the teacher is to provide an estimate for the unknown Bayes conditional probability distribution (BCPD) to be used i…
Conditional Mutual Information Constrained Deep Learning for Classification
En-Hui Yang, Shayan Mohajer Hamidi, Linfeng Ye +2
The concepts of conditional mutual information (CMI) and normalized conditional mutual information (NCMI) are introduced to measure the concentration and separation performance of…