Syllable-Based Sequence-to-Sequence Speech Recognition with the Transformer in Mandarin Chinese

Abstract

Sequence-to-sequence attention-based models have recently shown very promising results on automatic speech recognition (ASR) tasks, which integrate an acoustic, pronunciation and language model into a single neural network. In these models, the Transformer, a new sequence-to-sequence attention-based model relying entirely on self-attention without using RNNs or convolutions, achieves a new single-model state-of-the-art BLEU on neural machine translation (NMT) tasks. Since the outstanding performance of the Transformer, we extend it to speech and concentrate on it as the basic architecture of sequence-to-sequence attention-based model on Mandarin Chinese ASR tasks. Furthermore, we investigate a comparison between syllable based model and context-independent phoneme (CI-phoneme) based model with the Transformer in Mandarin Chinese. Additionally, a greedy cascading decoder with the Transformer is proposed for mapping CI-phoneme sequences and syllable sequences into word sequences. Experiments on HKUST datasets demonstrate that syllable based model with the Transformer performs better than CI-phoneme based counterpart, and achieves a character error rate (CER) of \emph{$28.77\%$}, which is competitive to the state-of-the-art CER of $28.0\%$ by the joint CTC-attention based encoder-decoder network.

arXiv commentaccepted by INTERSPEECH2018

References in corpus (3)

• Adam: A Method for Stochastic Optimization
• TensorFlow: Large-Scale Machine Learning on Heterogeneous Distributed Systems
• Attention-Based Models for Speech Recognition

Cited by in corpus (14)

• Transformer-based Acoustic Modeling for Hybrid Speech Recognition
• Root Mean Square Layer Normalization
• Multilingual End-to-End Speech Recognition with A Single Transformer on Low-Resource Languages
• Towards Data Distillation for End-to-end Spoken Conversational Question Answering
• Emformer: Efficient Memory Transformer Based Acoustic Model For Low Latency Streaming Speech Recognition
• Masked Pre-trained Encoder base on Joint CTC-Transformer
• DFSMN-SAN with Persistent Memory Model for Automatic Speech Recognition
• Weak-Attention Suppression For Transformer Based Speech Recognition
• Learn Spelling from Teachers: Transferring Knowledge from Language Models to Sequence-to-Sequence Speech Recognition
• Transformer with Bidirectional Decoder for Speech Recognition
• Learning Noise-Invariant Representations for Robust Speech Recognition
• Cross-task pre-training for on-device acoustic scene classification
• Flexi-Transducer: Optimizing Latency, Accuracy and Compute forMulti-Domain On-Device Scenarios
• Exploring Lexicon-Free Modeling Units for End-to-End Korean and Korean-English Code-Switching Speech Recognition