LibriMix: An Open-Source Dataset for Generalizable Speech Separation

Abstract

In recent years, wsj0-2mix has become the reference dataset for single-channel speech separation. Most deep learning-based speech separation models today are benchmarked on it. However, recent studies have shown important performance drops when models trained on wsj0-2mix are evaluated on other, similar datasets. To address this generalization issue, we created LibriMix, an open-source alternative to wsj0-2mix, and to its noisy extension, WHAM!. Based on LibriSpeech, LibriMix consists of two- or three-speaker mixtures combined with ambient noise samples from WHAM!. Using Conv-TasNet, we achieve competitive performance on all LibriMix versions. In order to fairly evaluate across datasets, we introduce a third test set based on VCTK for speech and WHAM! for noise. Our experiments show that the generalization error is smaller for models trained with LibriMix than with WHAM!, in both clean and noisy conditions. Aiming towards evaluation in more realistic, conversation-like scenarios, we also release a sparsely overlapping version of LibriMix's test set.

submitted to INTERSPEECH 2020

References in corpus (3)

• Adam: A Method for Stochastic Optimization
• Conv-TasNet: Surpassing Ideal Time-Frequency Magnitude Masking for Speech Separation
• Analysis of Deep Clustering as Preprocessing for Automatic Speech Recognition of Sparsely Overlapping Speech

Cited by in corpus (27)

• SpeechBrain: A General-Purpose Speech Toolkit
• Unsupervised Sound Separation Using Mixture Invariant Training
• SUPERB: Speech processing Universal PERformance Benchmark
• Augmented Datasheets for Speech Datasets and Ethical Decision-Making
• GWA: A Large High-Quality Acoustic Dataset for Audio Processing
• Objective and subjective evaluation of speech enhancement methods in the UDASE task of the 7th CHiME challenge
• The CHiME-7 UDASE task: Unsupervised domain adaptation for conversational speech enhancement
• Speech Separation with Pretrained Frontend to Minimize Domain Mismatch
• Towards Listening to 10 People Simultaneously: An Efficient Permutation Invariant Training of Audio Source Separation Using Sinkhorn's Algorithm
• Stepwise-Refining Speech Separation Network via Fine-Grained Encoding in High-order Latent Domain
• MixCycle: Unsupervised Speech Separation via Cyclic Mixture Permutation Invariant Training
• Sparsely Overlapped Speech Training in the Time Domain: Joint Learning of Target Speech Separation and Personal VAD Benefits
• Continuous Speech Separation Using Speaker Inventory for Long Multi-talker Recording
• End-to-End Integration of Speech Separation and Voice Activity Detection for Low-Latency Diarization of Telephone Conversations
• ESPnet-se: end-to-end speech enhancement and separation toolkit designed for asr integration
• Speech Representation Learning Through Self-supervised Pretraining And Multi-task Finetuning
• Rethinking the Separation Layers in Speech Separation Networks
• Target Speaker Verification with Selective Auditory Attention for Single and Multi-talker Speech
• TransMask: A Compact and Fast Speech Separation Model Based on Transformer
• Causal Signal-Based DCCRN with Overlapped-Frame Prediction for Online Speech Enhancement
• Multi-Scale Temporal Convolution Network for Classroom Voice Detection
• Improving Reverberant Speech Separation with Multi-stage Training and Curriculum Learning
• Stabilizing Label Assignment for Speech Separation by Self-supervised Pre-training
• Contrastive Separative Coding for Self-supervised Representation Learning
• Tune-In: Training Under Negative Environments with Interference for Attention Networks Simulating Cocktail Party Effect
• On permutation invariant training for speech source separation
• Refining DNN-based Mask Estimation using CGMM-based EM Algorithm for Multi-channel Noise Reduction