Representative Teacher Keys for Knowledge Distillation Model Compression Based on Attention Mechanism for Image Classification

Jun-Teng Yang, Sheng-Che Kao, Scott C. -H. Huang

With the improvement of AI chips (e.g., GPU, TPU, and NPU) and the fast development of the Internet of Things (IoT), some robust deep neural networks (DNNs) are usually composed of…