BeamLearning: an end-to-end Deep Learning approach for the angular localization of sound sources using raw multichannel acoustic pressure data
arXiv:2104.13347 · doi:10.1121/10.0005046
Abstract
Sound sources localization using multichannel signal processing has been a subject of active research for decades. In recent years, the use of deep learning in audio signal processing has allowed to drastically improve performances for machine hearing. This has motivated the scientific community to also develop machine learning strategies for source localization applications. In this paper, we present BeamLearning, a multi-resolution deep learning approach that allows to encode relevant information contained in unprocessed time domain acoustic signals captured by microphone arrays. The use of raw data aims at avoiding simplifying hypothesis that most traditional model-based localization methods rely on. Benefits of its use are shown for realtime sound source 2D-localization tasks in reverberating and noisy environments. Since supervised machine learning approaches require large-sized, physically realistic, precisely labelled datasets, we also developed a fast GPU-based computation of room impulse responses using fractional delays for image source models. A thorough analysis of the network representation and extensive performance tests are carried out using the BeamLearning network with synthetic and experimental datasets. Obtained results demonstrate that the BeamLearning approach significantly outperforms the wideband MUSIC and SRP-PHAT methods in terms of localization accuracy and computational efficiency in presence of heavy measurement noise and reverberation.
arXiv commentThe following article has been submitted to the special issue on Machine Learning in Acoustics in JASA. After it is published, it will be found at http://asa.scitation.org/journal/jas
References in corpus (16)
- Adam: A Method for Stochastic Optimization
- Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift
- Machine learning in acoustics: theory and applications
- Pyroomacoustics: A Python package for audio room simulations and array processing algorithms
- Sound Event Localization and Detection of Overlapping Sources Using Convolutional Recurrent Neural Networks
- Multi-Speaker DOA Estimation Using Deep Convolutional Networks Trained with Noise Signals
- Broadband DOA estimation using Convolutional neural networks trained with noise signals
- Depthwise Separable Convolutions for Neural Machine Translation
- Batch Renormalization: Towards Reducing Minibatch Dependence in Batch-Normalized Models
- Deep Neural Networks for Multiple Speaker Detection and Localization
- Exploiting Deep Neural Networks and Head Movements for Robust Binaural Localisation of Multiple Sources in Reverberant Environments
- Towards End-to-End Acoustic Localization using Deep Learning: from Audio Signal to Source Position Coordinates
- Acoustic Space Learning for Sound Source Separation and Localization on Binaural Manifolds
- Quaternion Convolutional Neural Networks for Detection and Localization of 3D Sound Events
- Regression and Classification for Direction-of-Arrival Estimation with Convolutional Recurrent Neural Networks
- Deep Residual Network for Sound Source Localization in the Time Domain