Insertion-Based Modeling for End-to-End Automatic Speech Recognition

Abstract

End-to-end (E2E) models have gained attention in the research field of automatic speech recognition (ASR). Many E2E models proposed so far assume left-to-right autoregressive generation of an output token sequence except for connectionist temporal classification (CTC) and its variants. However, left-to-right decoding cannot consider the future output context, and it is not always optimal for ASR. One of the non-left-to-right models is known as non-autoregressive Transformer (NAT) and has been intensively investigated in the area of neural machine translation (NMT) research. One NAT model, mask-predict, has been applied to ASR but the model needs some heuristics or additional component to estimate the length of the output token sequence. This paper proposes to apply another type of NAT called insertion-based models, that were originally proposed for NMT, to ASR tasks. Insertion-based models solve the above mask-predict issues and can generate an arbitrary generation order of an output sequence. In addition, we introduce a new formulation of joint training of the insertion-based models and CTC. This formulation reinforces CTC by making it dependent on insertion-based token generation in a non-autoregressive manner. We conducted experiments on three public benchmarks and achieved competitive performance to strong autoregressive Transformer with a similar decoding condition.

arXiv commentINTERSPEECH 2020

References in corpus (10)

• SpecAugment: A Simple Data Augmentation Method for Automatic Speech Recognition
• Deep Speech 2: End-to-End Speech Recognition in English and Mandarin
• Attention-Based Models for Speech Recognition
• A Comparative Study on Transformer vs RNN in Speech Applications
• Non-Autoregressive Neural Machine Translation
• Improved training of end-to-end attention models for speech recognition
• Insertion Transformer: Flexible Sequence Generation via Insertion Operations
• Self-Attention Networks for Connectionist Temporal Classification in Speech Recognition
• KERMIT: Generative Insertion-Based Modeling for Sequences
• Imputer: Sequence Modelling via Imputation and Dynamic Programming