Knowledge Distillation via Weighted Ensemble of Teaching Assistants

Durga Prasad Ganta, Himel Das Gupta, Victor S. Sheng

Knowledge distillation in machine learning is the process of transferring knowledge from a large model called the teacher to a smaller model called the student. Knowledge distillat…