Adapting Models to Signal Degradation using Distillation

Jong-Chyi Su, Subhransu Maji

Model compression and knowledge distillation have been successfully applied for cross-architecture and cross-domain transfer learning. However, a key requirement is that training e…