Multi-Level Feature Distillation of Joint Teachers Trained on Distinct Image Datasets

Adrian Iordache, Bogdan Alexe, Radu Tudor Ionescu

We propose a novel teacher-student framework to distill knowledge from multiple teachers trained on distinct datasets. Each teacher is first trained from scratch on its own dataset…