Whisper-MLA: Reducing GPU Memory Consumption of ASR Models based on MHA2MLA Conversion

Sen Zhang, Jianguo Wei, Wenhuan Lu +6

The Transformer-based Whisper model has achieved state-of-the-art performance in Automatic Speech Recognition (ASR). However, its Multi-Head Attention (MHA) mechanism results in si…