Heterogeneous Target Speech Separation
arXiv:2204.03594 · doi:10.21437/Interspeech.2022-46
Abstract
We introduce a new paradigm for single-channel target source separation where the sources of interest can be distinguished using non-mutually exclusive concepts (e.g., loudness, gender, language, spatial location, etc). Our proposed heterogeneous separation framework can seamlessly leverage datasets with large distribution shifts and learn cross-domain representations under a variety of concepts used as conditioning. Our experiments show that training separation models with heterogeneous conditions facilitates the generalization to new concepts with unseen out-of-domain data while also performing substantially higher than single-domain specialist models. Notably, such training leads to more robust learning of new harder source separation discriminative concepts and can yield improvements over permutation invariant training with oracle source selection. We analyze the intrinsic behavior of source separation training with heterogeneous metadata and propose ways to alleviate emerging problems with challenging separation conditions. We release the collection of preparation recipes for all datasets used to further promote research towards this challenging task.
arXiv commentSubmitted to Interspeech 2022
References in corpus (10)
- Neural Machine Translation by Jointly Learning to Align and Translate
- Variational Inference with Normalizing Flows
- FastSpeech: Fast, Robust and Controllable Text to Speech
- FastSpeech 2: Fast and High-Quality End-to-End Text to Speech
- Transfer Learning from Speaker Verification to Multispeaker Text-To-Speech Synthesis
- Glow-TTS: A Generative Flow for Text-to-Speech via Monotonic Alignment Search
- AISHELL-3: A Multi-speaker Mandarin TTS Corpus and the Baselines
- Meta-StyleSpeech : Multi-Speaker Adaptive Text-to-Speech Generation
- JVS corpus: free Japanese multi-speaker voice corpus
- Cross-lingual Multi-speaker Text-to-speech Synthesis for Voice Cloning without Using Parallel Corpus for Unseen Speakers