What Actually Makes Correlation-Based SSL Distillation Noise-Robust? A Mechanistic Correction

Fabian Ritter-Gutierrez, Nancy F. Chen, Eng Siong Chng

Self-supervised learning (SSL) speech models are accurate but large. Knowledge distillation compresses them, but the student loses the teacher's noise robustness. Correlation-based…