Knowledge Distillation for Small-footprint Highway Networks

Liang Lu, Michelle Guo, Steve Renals

Deep learning has significantly advanced state-of-the-art of speech recognition in the past few years. However, compared to conventional Gaussian mixture acoustic models, neural ne…