Investigation of Practical Aspects of Single Channel Speech Separation for ASR

Abstract

Speech separation has been successfully applied as a frontend processing module of conversation transcription systems thanks to its ability to handle overlapped speech and its flexibility to combine with downstream tasks such as automatic speech recognition (ASR). However, a speech separation model often introduces target speech distortion, resulting in a sub-optimum word error rate (WER). In this paper, we describe our efforts to improve the performance of a single channel speech separation system. Specifically, we investigate a two-stage training scheme that firstly applies a feature level optimization criterion for pretraining, followed by an ASR-oriented optimization criterion using an end-to-end (E2E) speech recognition model. Meanwhile, to keep the model light-weight, we introduce a modified teacher-student learning technique for model compression. By combining those approaches, we achieve a absolute average WER improvement of 2.70% and 0.77% using models with less than 10M parameters compared with the previous state-of-the-art results on the LibriCSS dataset for utterance-wise evaluation and continuous evaluation, respectively

arXiv commentAccepted by Interspeech 2021

References in corpus (16)

• Conv-TasNet: Surpassing Ideal Time-Frequency Magnitude Masking for Speech Separation
• Conformer: Convolution-augmented Transformer for Speech Recognition
• Speaker-independent Speech Separation with Deep Attractor Network
• DCCRN: Deep Complex Convolution Recurrent Network for Phase-Aware Speech Enhancement
• Multi-talker ASR for an unknown number of sources: Joint training of source counting, separation and ASR
• End-to-end training of time domain audio separation and recognition
• Semantic Mask for Transformer based End-to-End Speech Recognition
• SpEx+: A Complete Time Domain Speaker Extraction Network
• Advances in Online Audio-Visual Meeting Transcription
• Ultra Fast Speech Separation Model with Teacher Student Learning
• Guided Source Separation Meets a Strong ASR Backend: Hitachi/Paderborn University Joint Investigation for Dinner Party ASR
• Continuous Speech Separation with Conformer
• Recall and Learn: Fine-tuning Deep Pretrained Language Models with Less Forgetting
• VoiceFilter-Lite: Streaming Targeted Voice Separation for On-Device Speech Recognition
• Building state-of-the-art distant speech recognition using the CHiME-4 challenge with a setup of speech enhancement baseline
• An End-to-end Architecture of Online Multi-channel Speech Separation