Multi-Speaker DOA Estimation Using Deep Convolutional Networks Trained with Noise Signals

Abstract

Supervised learning based methods for source localization, being data driven, can be adapted to different acoustic conditions via training and have been shown to be robust to adverse acoustic environments. In this paper, a convolutional neural network (CNN) based supervised learning method for estimating the direction-of-arrival (DOA) of multiple speakers is proposed. Multi-speaker DOA estimation is formulated as a multi-class multi-label classification problem, where the assignment of each DOA label to the input feature is treated as a separate binary classification problem. The phase component of the short-time Fourier transform (STFT) coefficients of the received microphone signals are directly fed into the CNN, and the features for DOA estimation are learnt during training. Utilizing the assumption of disjoint speaker activity in the STFT domain, a novel method is proposed to train the CNN with synthesized noise signals. Through experimental evaluation with both simulated and measured acoustic impulse responses, the ability of the proposed DOA estimation approach to adapt to unseen acoustic conditions and its robustness to unseen noise type is demonstrated. Through additional empirical investigation, it is also shown that with an array of M microphones our proposed framework yields the best localization performance with M-1 convolution layers. The ability of the proposed method to accurately localize speakers in a dynamic acoustic scenario with varying number of sources is also shown.

References in corpus (6)

• Adam: A Method for Stochastic Optimization
• Broadband DOA estimation using Convolutional neural networks trained with noise signals
• Deep Neural Networks for Multiple Speaker Detection and Localization
• Exploiting Deep Neural Networks and Head Movements for Robust Binaural Localisation of Multiple Sources in Reverberant Environments
• Classification vs. Regression in Supervised Learning for Single Channel Speaker Count Estimation
• Multi-Speaker Localization Using Convolutional Neural Network Trained with Noise

Cited by in corpus (34)

• Machine learning in acoustics: theory and applications
• Deep Networks for Direction-of-Arrival Estimation in Low SNR
• A Survey of Sound Source Localization with Deep Learning Methods
• Overview and Evaluation of Sound Event Localization and Detection in DCASE 2019
• DA-MUSIC: Data-Driven DoA Estimation via Deep Augmented MUSIC Algorithm
• Multi-modal Multi-channel Target Speech Separation
• Robust Sound Source Tracking Using SRP-PHAT and 3D Convolutional Neural Networks
• Complex ResNet Aided DoA Estimation for Near-Field MIMO Systems
• A Machine Learning Approach to DoA Estimation and Model Order Selection for Antenna Arrays with Subarray Sampling
• A Dataset of Reverberant Spatial Sound Scenes with Moving Sources for Sound Event Localization and Detection
• Direction of Arrival Estimation of Sound Sources Using Icosahedral CNNs
• Semi-supervised source localization with deep generative modeling
• Multi-Source DOA Estimation through Pattern Recognition of the Modal Coherence of a Reverberant Soundfield
• BeamLearning: an end-to-end Deep Learning approach for the angular localization of sound sources using raw multichannel acoustic pressure data
• Signal-Aware Direction-of-Arrival Estimation Using Attention Mechanisms
• Audio scene monitoring using redundant ad-hoc microphone array networks
• A Review on Sound Source Localization in Robotics: Focusing on Deep Learning Methods
• Multi-scale aggregation of phase information for reducing computational cost of CNN based DOA estimation
• DBNET: DOA-driven beamforming network for end-to-end farfield sound source separation
• SSLIDE: Sound Source Localization for Indoors based on Deep Learning
• Directional ASR: A New Paradigm for E2E Multi-Speaker Speech Recognition with Source Localization
• FCN Approach for Dynamically Locating Multiple Speakers
• Multi-Microphone Complex Spectral Mapping for Speech Dereverberation
• Multi-target DoA Estimation with an Audio-visual Fusion Mechanism
• Efficient Training Data Generation for Phase-Based DOA Estimation
• A Comparative Study of Multilateration Methods for Single-Source Localization in Distributed Audio
• Refinement of Direction of Arrival Estimators by Majorization-Minimization Optimization on the Array Manifold
• U-net Based Direct-path Dominance Test for Robust Direction-of-arrival Estimation
• Deep Multi-task Network for Delay Estimation and Echo Cancellation
• ChainNet: Neural Network-Based Successive Spectral Analysis
• Multi-microphone Complex Spectral Mapping for Utterance-wise and Continuous Speech Separation
• Completing Sets of Prototype Transfer Functions for Subspace-based Direction of Arrival Estimation of Multiple Speakers
• Study of speaker localization with binaural microphone array incorporating auditory filters and lateral angle estimation
• SLoClas: A Database for Joint Sound Localization and Classification