Unsupervised Deep Clustering for Source Separation: Direct Learning from Mixtures using Spatial Information

Abstract

We present a monophonic source separation system that is trained by only observing mixtures with no ground truth separation information. We use a deep clustering approach which trains on multi-channel mixtures and learns to project spectrogram bins to source clusters that correlate with various spatial features. We show that using such a training process we can obtain separation performance that is as good as making use of ground truth separation information. Once trained, this system is capable of performing sound separation on monophonic inputs, despite having learned how to do so using multi-channel recordings.

Submitted to ICASSP 2019 (v1: November 5th 2018)

Cited by in corpus (13)

• Unsupervised Sound Separation Using Mixture Invariant Training
• Two-Step Sound Source Separation: Training on Learned Latent Targets
• Into the Wild with AudioScope: Unsupervised Audio-Visual Separation of On-Screen Sounds
• The Cone of Silence: Speech Separation by Localization
• Separate but Together: Unsupervised Federated Learning for Speech Enhancement from Non-IID Data
• Microphone Array Signal Processing and Deep Learning for Speech Enhancement
• Bootstrapping deep music separation from primitive auditory grouping principles
• Improving On-Screen Sound Separation for Open-Domain Videos with Audio-Visual Self-Attention
• Unsupervised Training for Deep Speech Source Separation with Kullback-Leibler Divergence Based Probabilistic Loss Function
• OtoWorld: Towards Learning to Separate by Learning to Move
• Sparse Mixture of Local Experts for Efficient Speech Enhancement
• Far-Field Automatic Speech Recognition
• Deep Bayesian Unsupervised Source Separation Based on a Complex Gaussian Mixture Model