47 citations · 57 across the 9 of their papers we have counts for
4 papers · 1 filter
GUIDE: Guided Initialization and Distillation of Embeddings
Khoa Trinh, Gaurav Menghani, Erik Vee
Algorithmic efficiency techniques such as distillation (\cite{hinton2015distillation}) are useful in improving model quality without increasing serving costs, provided a larger tea…
SLaM: Student-Label Mixing for Distillation with Unlabeled Examples
Vasilis Kontonis, Fotis Iliopoulos, Khoa Trinh +3
Knowledge distillation with unlabeled examples is a powerful training paradigm for generating compact and lightweight student models in applications where the amount of labeled dat…
Weighted Distillation with Unlabeled Examples
Fotis Iliopoulos, Vasilis Kontonis, Cenk Baykal +3
Distillation with unlabeled examples is a popular and powerful method for training deep neural networks in settings where the amount of labeled data is limited: A large ''teacher''…
Robust Active Distillation
Cenk Baykal, Khoa Trinh, Fotis Iliopoulos +2
Distilling knowledge from a large teacher model to a lightweight one is a widely successful approach for generating compact, powerful models in the semi-supervised learning setting…