Analysis of Deep Clustering as Preprocessing for Automatic Speech Recognition of Sparsely Overlapping Speech

Abstract

Significant performance degradation of automatic speech recognition (ASR) systems is observed when the audio signal contains cross-talk. One of the recently proposed approaches to solve the problem of multi-speaker ASR is the deep clustering (DPCL) approach. Combining DPCL with a state-of-the-art hybrid acoustic model, we obtain a word error rate (WER) of 16.5 % on the commonly used wsj0-2mix dataset, which is the best performance reported thus far to the best of our knowledge. The wsj0-2mix dataset contains simulated cross-talk where the speech of multiple speakers overlaps for almost the entire utterance. In a more realistic ASR scenario the audio signal contains significant portions of single-speaker speech and only part of the signal contains speech of multiple competing speakers. This paper investigates obstacles of applying DPCL as a preprocessing method for ASR in such a scenario of sparsely overlapping speech. To this end we present a data simulation approach, closely related to the wsj0-2mix dataset, generating sparsely overlapping speech datasets of arbitrary overlap ratio. The analysis of applying DPCL to sparsely overlapping speech is an important interim step between the fully overlapping datasets like wsj0-2mix and more realistic ASR datasets, such as CHiME-5 or AMI.

References in corpus (4)

• Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift
• The Microsoft 2017 Conversational Speech Recognition System
• Deep attractor network for single-microphone speaker separation
• The CAPIO 2017 Conversational Speech Recognition System

Cited by in corpus (13)

• LibriMix: An Open-Source Dataset for Generalizable Speech Separation
• On End-to-end Multi-channel Time Domain Speech Separation in Reverberant Environments
• Multi-talker ASR for an unknown number of sources: Joint training of source counting, separation and ASR
• End-to-end training of time domain audio separation and recognition
• Compositional Clustering: Applications to Multi-Label Object Recognition and Speaker Identification
• End-to-End Multi-speaker Speech Recognition with Transformer
• A Survey on Spoken Italian Datasets and Corpora
• Sequence to Multi-Sequence Learning via Conditional Chain Mapping for Mixture Signals
• Streaming Multi-speaker ASR with RNN-T
• Multi-Speaker ASR Combining Non-Autoregressive Conformer CTC and Conditional Speaker Chain
• MIMO-SPEECH: End-to-End Multi-Channel Multi-Speaker Speech Recognition
• Utterance Clustering Using Stereo Audio Channels
• A Study of the Scale Invariant Signal to Distortion Ratio in Speech Separation with Noisy References