An investigation of phone-based subword units for end-to-end speech recognition

Abstract

Phones and their context-dependent variants have been the standard modeling units for conventional speech recognition systems, while characters and subwords have demonstrated their effectiveness for end-to-end recognition systems. We investigate the use of phone-based subwords, in particular, byte pair encoder (BPE), as modeling units for end-to-end speech recognition. In addition, we also developed multi-level language model-based decoding algorithms based on a pronunciation dictionary. Besides the use of the lexicon, which is easily available, our system avoids the need of additional expert knowledge or processing steps from conventional systems. Experimental results show that phone-based BPEs tend to yield more accurate recognition systems than the character-based counterpart. In addition, further improvement can be obtained with a novel one-pass joint beam search decoder, which efficiently combines phone- and character-based BPE systems. For Switchboard, our phone-based BPE system achieves 6.8\%/14.4\% word error rate (WER) on the Switchboard/CallHome portion of the test set while joint decoding achieves 6.3\%/13.3\% WER. On Fisher + Switchboard, joint decoding leads to 4.9\%/9.5\% WER, setting new milestones for telephony speech recognition.

arXiv commentInterspeech 2020 final version. Implementation for reproducing the results can be found at: https://github.com/salesforce/transformerasr

References in corpus (8)

• SpecAugment: A Simple Data Augmentation Method for Automatic Speech Recognition
• Deep Speech 2: End-to-End Speech Recognition in English and Mandarin
• A Comparative Study on Transformer vs RNN in Speech Applications
• On Using Monolingual Corpora in Neural Machine Translation
• Improved training of end-to-end attention models for speech recognition
• Wav2Letter: an End-to-End ConvNet-based Speech Recognition System
• The CAPIO 2017 Conversational Speech Recognition System
• Transformer based Grapheme-to-Phoneme Conversion

Cited by in corpus (5)

• A New Training Pipeline for an Improved Neural Transducer
• Pushing the Limits of Non-Autoregressive Speech Recognition
• A systematic comparison of grapheme-based vs. phoneme-based label units for encoder-decoder-attention models
• KoSpeech: Open-Source Toolkit for End-to-End Korean Speech Recognition
• Phoneme Based Neural Transducer for Large Vocabulary Speech Recognition