Two-Step Sound Source Separation: Training on Learned Latent Targets

Abstract

In this paper, we propose a two-step training procedure for source separation via a deep neural network. In the first step we learn a transform (and it's inverse) to a latent space where masking-based separation performance using oracles is optimal. For the second step, we train a separation module that operates on the previously learned space. In order to do so, we also make use of a scale-invariant signal to distortion ratio (SI-SDR) loss function that works in the latent space, and we prove that it lower-bounds the SI-SDR in the time domain. We run various sound separation experiments that show how this approach can obtain better performance as compared to systems that learn the transform and the separation module jointly. The proposed methodology is general enough to be applicable to a large class of neural network end-to-end separation systems.

Submitted to ICASSP 2020

References in corpus (1)

• Divide and Conquer: A Deep CASA Approach to Talker-independent Monaural Speaker Separation

Cited by in corpus (12)

• Asteroid: the PyTorch-based audio source separation toolkit for researchers
• Compute and memory efficient universal sound source separation
• Learning Semantic-Agnostic and Spatial-Aware Representation for Generalizable Visual-Audio Navigation
• Attention is All You Need in Speech Separation
• Stepwise-Refining Speech Separation Network via Fine-Grained Encoding in High-order Latent Domain
• RemixIT: Continual self-training of speech enhancement models via bootstrapped remixing
• Latent Iterative Refinement for Modular Source Separation
• Improving On-Screen Sound Separation for Open-Domain Videos with Audio-Visual Self-Attention
• Revisiting Representation Learning for Singing Voice Separation with Sinkhorn Distances
• Unified Gradient Reweighting for Model Biasing with Applications to Source Separation
• Multi-channel Speech Enhancement with 2-D Convolutional Time-frequency Domain Features and a Pre-trained Acoustic Model
• On permutation invariant training for speech source separation