Complete and separate: Conditional separation with missing target source attribute completion

Abstract

Recent approaches in source separation leverage semantic information about their input mixtures and constituent sources that when used in conditional separation models can achieve impressive performance. Most approaches along these lines have focused on simple descriptions, which are not always useful for varying types of input mixtures. In this work, we present an approach in which a model, given an input mixture and partial semantic information about a target source, is trained to extract additional semantic data. We then leverage this pre-trained model to improve the separation performance of an uncoupled multi-conditional separation network. Our experiments demonstrate that the separation performance of this multi-conditional model is significantly improved, approaching the performance of an oracle model with complete semantic information. Furthermore, our approach achieves performance levels that are comparable to those of the best performing specialized single conditional models, thus providing an easier to use alternative.

arXiv commentAccepted to IEEE Workshop on Applications of Signal Processing to Audio and Acoustics (WASPAA) 2023

References in corpus (6)

• Adam: A Method for Stochastic Optimization
• Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift
• ECAPA-TDNN: Emphasized Channel Attention, Propagation and Aggregation in TDNN Based Speaker Verification
• Improving Universal Sound Separation Using Sound Classification
• Conditioned-U-Net: Introducing a Control Mechanism in the U-Net for Multiple Source Separations
• CLIPSep: Learning Text-queried Sound Separation with Noisy Unlabeled Videos