2 papers
cs.LG2025
WIND: Accelerated RNN-T Decoding with Windowed Inference for Non-blank Detection
Hainan Xu, Vladimir Bataev, Lilit Grigoryan +1
We propose Windowed Inference for Non-blank Detection (WIND), a novel strategy that significantly accelerates RNN-T inference without compromising model accuracy. During model infe…
cs.LG2024
Speed of Light Exact Greedy Decoding for RNN-T Speech Recognition Models on GPU
Daniel Galvez, Vladimir Bataev, Hainan Xu +1
The vast majority of inference time for RNN Transducer (RNN-T) models today is spent on decoding. Current state-of-the-art RNN-T decoding implementations leave the GPU idle ~80% of…