Fast Speech Foundation Model Distillation Using Interleaved Stacking

Eungbeom Kim, Kyogu Lee

Distilling a large speech foundation model (SFM) into an efficient student model has been successfully applied to low-resource environments. Although distillation reduces inference…