Synthetic data generation method for data-free knowledge distillation in regression neural networks

Tianxun Zhou, Keng-Hwee Chiam

Knowledge distillation is the technique of compressing a larger neural network, known as the teacher, into a smaller neural network, known as the student, while still trying to mai…