RdimKD: Generic Distillation Paradigm by Dimensionality Reduction

Yi Guo, Yiqian He, Xiaoyang Li +4

Knowledge Distillation (KD) emerges as one of the most promising compression technologies to run advanced deep neural networks on resource-limited devices. In order to train a smal…