Improved Conformer-based End-to-End Speech Recognition Using Neural Architecture Search

Abstract

Recently neural architecture search(NAS) has been successfully used in image classification, natural language processing, and automatic speech recognition(ASR) tasks for finding the state-of-the-art(SOTA) architectures than those human-designed architectures. NAS can derive a SOTA and data-specific architecture over validation data from a pre-defined search space with a search algorithm. Inspired by the success of NAS in ASR tasks, we propose a NAS-based ASR framework containing one search space and one differentiable search algorithm called Differentiable Architecture Search(DARTS). Our search space follows the convolution-augmented transformer(Conformer) backbone, which is a more expressive ASR architecture than those used in existing NAS-based ASR frameworks. To improve the performance of our method, a regulation method called Dynamic Search Schedule(DSS) is employed. On a widely used Mandarin benchmark AISHELL-1, our best-searched architecture outperforms the baseline Conform model significantly with about 11% CER relative improvement, and our method is proved to be pretty efficient by the search cost comparisons.

submitted to INTERSPEECH 2021

References in corpus (7)

• Neural Architecture Search with Reinforcement Learning
• Sequence Transduction with Recurrent Neural Networks
• Conformer: Convolution-augmented Transformer for Speech Recognition
• The Evolved Transformer
• ContextNet: Improving Convolutional Neural Networks for Automatic Speech Recognition with Global Context
• Efficient Neural Architecture Search for End-to-end Speech Recognition via Straight-Through Gradients
• Conv-Transformer Transducer: Low Latency, Low Frame Rate, Streamable End-to-End Speech Recognition