Task-Agnostic Structured Pruning of Speech Representation Models

Haoyu Wang, Siyuan Wang, Wei-Qiang Zhang +2

Self-supervised pre-trained models such as Wav2vec2, Hubert, and WavLM have been shown to significantly improve many speech tasks. However, their large memory and strong computatio…