Spike-Triggered Non-Autoregressive Transformer for End-to-End Speech Recognition

Abstract

Non-autoregressive transformer models have achieved extremely fast inference speed and comparable performance with autoregressive sequence-to-sequence models in neural machine translation. Most of the non-autoregressive transformers decode the target sequence from a predefined-length mask sequence. If the predefined length is too long, it will cause a lot of redundant calculations. If the predefined length is shorter than the length of the target sequence, it will hurt the performance of the model. To address this problem and improve the inference speed, we propose a spike-triggered non-autoregressive transformer model for end-to-end speech recognition, which introduces a CTC module to predict the length of the target sequence and accelerate the convergence. All the experiments are conducted on a public Chinese mandarin dataset AISHELL-1. The results show that the proposed model can accurately predict the length of the target sequence and achieve a competitive performance with the advanced transformers. What's more, the model even achieves a real-time factor of 0.0056, which exceeds all mainstream speech recognition models.

arXiv comment5 pages

References in corpus (7)

• SpecAugment: A Simple Data Augmentation Method for Automatic Speech Recognition
• Attention-Based Models for Speech Recognition
• Non-Autoregressive Neural Machine Translation
• Self-Attention Networks for Connectionist Temporal Classification in Speech Recognition
• Self-Attention Transducers for End-to-End Speech Recognition
• FlowSeq: Non-Autoregressive Conditional Sequence Generation with Generative Flow
• End-to-End Non-Autoregressive Neural Machine Translation with Connectionist Temporal Classification

Cited by in corpus (8)

• Listen and Fill in the Missing Letters: Non-Autoregressive Transformer for Speech Recognition
• Transformer-based End-to-End Speech Recognition with Local Dense Synthesizer Attention
• Efficient conformer-based speech recognition with linear attention
• One In A Hundred: Select The Best Predicted Sequence from Numerous Candidates for Streaming Speech Recognition
• Transformer-based end-to-end speech recognition with residual Gaussian-based self-attention
• Decoupling recognition and transcription in Mandarin ASR
• Streaming End-to-End ASR based on Blockwise Non-Autoregressive Models
• Conformer-based End-to-end Speech Recognition With Rotary Position Embedding