Efficient Supernet Training with Orthogonal Softmax for Scalable ASR Model Compression

Jingjing Xu, Eugen Beck, Zijian Yang +1

ASR systems are deployed across diverse environments, each with specific hardware constraints. We use supernet training to jointly train multiple encoders of varying sizes, enablin…