Improving Reverberant Speech Separation with Multi-stage Training and Curriculum Learning

Abstract

We present a novel approach that improves the performance of reverberant speech separation. Our approach is based on an accurate geometric acoustic simulator (GAS) which generates realistic room impulse responses (RIRs) by modeling both specular and diffuse reflections. We also propose three training methods - pre-training, multi-stage training and curriculum learning that significantly improve separation quality in the presence of reverberation. We also demonstrate that mixing the synthetic RIRs with a small number of real RIRs during training enhances separation performance. We evaluate our approach on reverberant mixtures generated from real, recorded data (in several different room configurations) from the VOiCES dataset. Our novel approach (curriculum learning+pre-training+multi-stage training) results in a significant relative improvement over prior techniques based on image source method (ISM).

References in corpus (10)

• Adam: A Method for Stochastic Optimization
• Using Pre-Training Can Improve Model Robustness and Uncertainty
• Improved training of end-to-end attention models for speech recognition
• LibriMix: An Open-Source Dataset for Generalizable Speech Separation
• Building and Evaluation of a Real Room Impulse Response Dataset
• Voices Obscured in Complex Environmental Settings (VOICES) corpus
• Improving Reverberant Speech Training Using Diffuse Acoustic Simulation
• Curriculum Pre-training for End-to-End Speech Translation
• Time-Domain Speech Extraction with Spatial Information and Multi Speaker Conditioning Mechanism
• Interactive Sound Rendering on Mobile Devices using Ray-Parameterized Reverberation Filters