Multi-Channel Transformer Transducer for Speech Recognition
arXiv:2108.12953
Abstract
Multi-channel inputs offer several advantages over single-channel, to improve the robustness of on-device speech recognition systems. Recent work on multi-channel transformer, has proposed a way to incorporate such inputs into end-to-end ASR for improved accuracy. However, this approach is characterized by a high computational complexity, which prevents it from being deployed in on-device systems. In this paper, we present a novel speech recognition model, Multi-Channel Transformer Transducer (MCTT), which features end-to-end multi-channel training, low computation cost, and low latency so that it is suitable for streaming decoding in on-device speech recognition. In a far-field in-house dataset, our MCTT outperforms stagewise multi-channel models with transformer-transducer up to 6.01% relative WER improvement (WERR). In addition, MCTT outperforms the multi-channel transformer up to 11.62% WERR, and is 15.8 times faster in terms of inference speed. We further show that we can improve the computational cost of MCTT by constraining the future and previous context in attention computations.
References in corpus (9)
- Adam: A Method for Stochastic Optimization
- Sequence Transduction with Recurrent Neural Networks
- Conformer: Convolution-augmented Transformer for Speech Recognition
- Transformer-based Acoustic Modeling for Hybrid Speech Recognition
- Deep Long Short-Term Memory Adaptive Beamforming Networks For Multichannel Robust Speech Recognition
- Self-Attention Transducers for End-to-End Speech Recognition
- Transformer-Transducer: End-to-End Speech Recognition with Self-Attention
- Multi-Geometry Spatial Acoustic Modeling for Distant Speech Recognition
- Exploring Transformers for Large-Scale Speech Recognition