Towards End-to-End Acoustic Localization using Deep Learning: from Audio Signal to Source Position Coordinates

Abstract

This paper presents a novel approach for indoor acoustic source localization using microphone arrays and based on a Convolutional Neural Network (CNN). The proposed solution is, to the best of our knowledge, the first published work in which the CNN is designed to directly estimate the three dimensional position of an acoustic source, using the raw audio signal as the input information avoiding the use of hand crafted audio features. Given the limited amount of available localization data, we propose in this paper a training strategy based on two steps. We first train our network using semi-synthetic data, generated from close talk speech recordings, and where we simulate the time delays and distortion suffered in the signal that propagates from the source to the array of microphones. We then fine tune this network using a small amount of real data. Our experimental results show that this strategy is able to produce networks that significantly improve existing localization methods based on \textit{SRP-PHAT} strategies. In addition, our experiments show that our CNN method exhibits better resistance against varying gender of the speaker and different window sizes compared with the other methods.

arXiv comment18 pages, 3 figures, 8 tables

References in corpus (6)

• Broadband DOA estimation using Convolutional neural networks trained with noise signals
• Deep Neural Networks for Multiple Speaker Detection and Localization
• Exploiting Deep Neural Networks and Head Movements for Robust Binaural Localisation of Multiple Sources in Reverberant Environments
• Indoor Sound Source Localization with Probabilistic Neural Network
• A Geometrical-Statistical approach to outlier removal for TDOA measuments
• TDOA Matrices: Algebraic Properties and their Application to Robust Denoising with Missing Data

Cited by in corpus (13)

• A Survey of Sound Source Localization with Deep Learning Methods
• Robust Sound Source Tracking Using SRP-PHAT and 3D Convolutional Neural Networks
• Listening for Sirens: Locating and Classifying Acoustic Alarms in City Scenes
• Direction of Arrival Estimation of Sound Sources Using Icosahedral CNNs
• BeamLearning: an end-to-end Deep Learning approach for the angular localization of sound sources using raw multichannel acoustic pressure data
• Extending GCC-PHAT using Shift Equivariant Neural Networks
• A Deep Reinforcement Learning Approach to Audio-Based Navigation in a Multi-Speaker Environment
• Is the PPG signal chaotic?
• wav2pos: Sound Source Localization using Masked Autoencoders
• FCN Approach for Dynamically Locating Multiple Speakers
• Sound Event Localization and Detection using Squeeze-Excitation Residual CNNs
• Localization using Angle-of-Arrival Triangulation
• CNN-based Robust Sound Source Localization with SRP-PHAT for the Extreme Edge