Streaming Chunk-Aware Multihead Attention for Online End-to-End Speech Recognition

Abstract

Recently, streaming end-to-end automatic speech recognition (E2E-ASR) has gained more and more attention. Many efforts have been paid to turn the non-streaming attention-based E2E-ASR system into streaming architecture. In this work, we propose a novel online E2E-ASR system by using Streaming Chunk-Aware Multihead Attention(SCAMA) and a latency control memory equipped self-attention network (LC-SAN-M). LC-SAN-M uses chunk-level input to control the latency of encoder. As to SCAMA, a jointly trained predictor is used to control the output of encoder when feeding to decoder, which enables decoder to generate output in streaming manner. Experimental results on the open 170-hour AISHELL-1 and an industrial-level 20000-hour Mandarin speech recognition tasks show that our approach can significantly outperform the MoChA-based baseline system under comparable setup. On the AISHELL-1 task, our proposed method achieves a character error rate (CER) of 7.39%, to the best of our knowledge, which is the best published performance for online ASR.

arXiv commentsubmitted to INTERSPEECH2020

References in corpus (10)

• SpecAugment: A Simple Data Augmentation Method for Automatic Speech Recognition
• Attention-Based Models for Speech Recognition
• Sequence Transduction with Recurrent Neural Networks
• A Comparative Study on Transformer vs RNN in Speech Applications
• Online and Linear-Time Attention by Enforcing Monotonic Alignments
• Local Monotonic Attention Mechanism for End-to-End Speech and Language Processing
• Monotonic Chunkwise Attention
• Towards Online End-to-end Transformer Automatic Speech Recognition
• Low Latency End-to-End Streaming Speech Recognition with a Scout Network
• An Online Attention-based Model for Speech Recognition

Cited by in corpus (7)

• Unified Streaming and Non-streaming Two-pass End-to-end Model for Speech Recognition
• U2++: Unified Two-pass Bidirectional End-to-end Model for Speech Recognition
• Universal ASR: Unifying Streaming and Non-Streaming ASR Using a Single Encoder-Decoder Model
• WNARS: WFST based Non-autoregressive Streaming End-to-End Speech Recognition
• Darts-Conformer: Towards Efficient Gradient-Based Neural Architecture Search For End-to-End ASR
• Improving RNN transducer with normalized jointer network
• Extremely Low Footprint End-to-End ASR System for Smart Device