Inplace knowledge distillation with teacher assistant for improved training of flexible deep neural networks

Alexey Ozerov, Ngoc Duong

Deep neural networks (DNNs) have achieved great success in various machine learning tasks. However, most existing powerful DNN models are computationally expensive and memory deman…