17 citations · 39 across the 6 of their papers we have counts for
Showing 2020Show all
3 papers · 1 filter
cs.CV2020
Model Compression Using Optimal Transport
Suhas Lohit, Michael Jones
Model compression methods are important to allow for easier deployment of deep learning models in compute, memory and energy-constrained environments such as mobile phones. Knowled…
cs.CV2020★ 5 cited
Multi-head Knowledge Distillation for Model Compression
Huan Wang, Suhas Lohit, Michael Jones +1
Several methods of knowledge distillation have been developed for neural network compression. While they all use the KL divergence loss to align the soft outputs of the student mod…
cs.CV2020★ 17 cited
What Makes a "Good" Data Augmentation in Knowledge Distillation -- A Statistical Perspective
Huan Wang, Suhas Lohit, Mike Jones +1
Knowledge distillation (KD) is a general neural network training approach that uses a teacher model to guide the student model. Existing works mainly study KD from the network outp…