End-to-End Multi-Channel Speech Separation

Abstract

The end-to-end approach for single-channel speech separation has been studied recently and shown promising results. This paper extended the previous approach and proposed a new end-to-end model for multi-channel speech separation. The primary contributions of this work include 1) an integrated waveform-in waveform-out separation system in a single neural network architecture. 2) We reformulate the traditional short time Fourier transform (STFT) and inter-channel phase difference (IPD) as a function of time-domain convolution with a special kernel. 3) We further relaxed those fixed kernels to be learnable, so that the entire architecture becomes purely data-driven and can be trained from end-to-end. We demonstrate on the WSJ0 far-field speech separation task that, with the benefit of learnable spatial features, our proposed end-to-end multi-channel model significantly improved the performance of previous end-to-end single-channel method and traditional multi-channel methods.

submitted to interspeech 2019

References in corpus (7)

• Two-Stream Convolutional Networks for Action Recognition in Videos
• Conv-TasNet: Surpassing Ideal Time-Frequency Magnitude Masking for Speech Separation
• Convolutional Two-Stream Network Fusion for Video Action Recognition
• Speaker-independent Speech Separation with Deep Attractor Network
• Phasebook and Friends: Leveraging Discrete Representations for Source Separation
• Phase-aware Speech Enhancement with Deep Complex U-Net
• End-to-End Speech Separation with Unfolded Iterative Phase Reconstruction

Cited by in corpus (22)

• DCCRN: Deep Complex Convolution Recurrent Network for Phase-Aware Speech Enhancement
• On End-to-end Multi-channel Time Domain Speech Separation in Reverberant Environments
• Semantic Hearing: Programming Acoustic Scenes with Binaural Hearables
• End-to-end Microphone Permutation and Number Invariant Multi-channel Speech Separation
• Distributed speech separation in spatially unconstrained microphone arrays
• Real-time binaural speech separation with preserved spatial cues
• A Unified Framework for Speech Separation
• Enhancing End-to-End Multi-channel Speech Separation via Spatial Feature Learning
• Rethinking the Separation Layers in Speech Separation Networks
• Efficient Integration of Multi-channel Information for Speaker-independent Speech Separation
• Improving Channel Decorrelation for Multi-Channel Target Speech Extraction
• Audio-visual Multi-channel Integration and Recognition of Overlapped Speech
• Speaker Separation Using Speaker Inventories and Estimated Speech
• Multi-channel target speech extraction with channel decorrelation and target speaker adaptation
• Distortion-controlled Training for End-to-end Reverberant Speech Separation with Auxiliary Autoencoding Loss
• Inter-channel Conv-TasNet for multichannel speech enhancement
• Implicit Filter-and-sum Network for Multi-channel Speech Separation
• Mixed Precision DNN Qunatization for Overlapped Speech Separation and Recognition
• Speaker and Direction Inferred Dual-channel Speech Separation
• Attention-based scaling adaptation for target speech extraction
• On permutation invariant training for speech source separation
• Beam-Guided TasNet: An Iterative Speech Separation Framework with Multi-Channel Output