Large-scale Knowledge Distillation with Elastic Heterogeneous Computing Resources

Ji Liu, Daxiang Dong, Xi Wang +5

Although more layers and more parameters generally improve the accuracy of the models, such big models generally have high computational complexity and require big memory, which ex…