Stabilizing Label Assignment for Speech Separation by Self-supervised Pre-training

Abstract

Speech separation has been well developed, with the very successful permutation invariant training (PIT) approach, although the frequent label assignment switching happening during PIT training remains to be a problem when better convergence speed and achievable performance are desired. In this paper, we propose to perform self-supervised pre-training to stabilize the label assignment in training the speech separation model. Experiments over several types of self-supervised approaches, several typical speech separation models and two different datasets showed that very good improvements are achievable if a proper self-supervised approach is chosen.

arXiv commentInterspeech 2021

References in corpus (10)

• A Simple Framework for Contrastive Learning of Visual Representations
• Conv-TasNet: Surpassing Ideal Time-Frequency Magnitude Masking for Speech Separation
• Data-Efficient Image Recognition with Contrastive Predictive Coding
• Big Self-Supervised Models are Strong Semi-Supervised Learners
• Deep attractor network for single-microphone speaker separation
• Mockingjay: Unsupervised Speech Representation Learning with Deep Bidirectional Transformer Encoders
• TERA: Self-Supervised Learning of Transformer Encoder Representation for Speech
• Speaker-independent Speech Separation with Deep Attractor Network
• LibriMix: An Open-Source Dataset for Generalizable Speech Separation
• Dual-Path Transformer Network: Direct Context-Aware Modeling for End-to-End Monaural Speech Separation