An Efficient Method of Training Small Models for Regression Problems with Knowledge Distillation

Makoto Takamoto, Yusuke Morishita, Hitoshi Imaoka

Compressing deep neural network (DNN) models becomes a very important and necessary technique for real-world applications, such as deploying those models on mobile devices. Knowled…