Robust Sound Source Tracking Using SRP-PHAT and 3D Convolutional Neural Networks

Abstract

In this paper, we present a new single sound source DOA estimation and tracking system based on the well-known SRP-PHAT algorithm and a three-dimensional Convolutional Neural Network. It uses SRP-PHAT power maps as input features of a fully convolutional causal architecture that uses 3D convolutional layers to accurately perform the tracking of a sound source even in highly reverberant scenarios where most of the state of the art techniques fail. Unlike previous methods, since we do not use bidirectional recurrent layers and all our convolutional layers are causal in the time dimension, our system is feasible for real-time applications and it provides a new DOA estimation for each new SRP-PHAT map. To train the model, we introduce a new procedure to simulate random trajectories as they are needed during the training, equivalent to an infinite-size dataset with high flexibility to modify its acoustical conditions such as the reverberation time. We use both acoustical simulations on a large range of reverberation times and the actual recordings of the LOCATA dataset to prove the robustness of our system and its good performance even using low-resolution SRP-PHAT maps.

arXiv commentThis is a pre-print of an article published in IEEE/ACM Transactions on Audio Speech and Language Processing. The code to reproduce this work can be found in our GitHub repository: https://github.com/DavidDiazGuerra/Cross3D

References in corpus (9)

• PyTorch: An Imperative Style, High-Performance Deep Learning Library
• Sound Event Localization and Detection of Overlapping Sources Using Convolutional Recurrent Neural Networks
• Multi-Speaker DOA Estimation Using Deep Convolutional Networks Trained with Noise Signals
• Deep Neural Networks for Multiple Speaker Detection and Localization
• gpuRIR: A Python Library for Room Impulse Response Simulation with GPU Acceleration
• Towards End-to-End Acoustic Localization using Deep Learning: from Audio Signal to Source Position Coordinates
• Indoor Sound Source Localization with Probabilistic Neural Network
• Regression and Classification for Direction-of-Arrival Estimation with Convolutional Recurrent Neural Networks
• Theoretical Framework for the Optimization of Microphone Array Configuration for Humanoid Robot Audition

Cited by in corpus (10)

• A Survey of Sound Source Localization with Deep Learning Methods
• gpuRIR: A Python Library for Room Impulse Response Simulation with GPU Acceleration
• Direction of Arrival Estimation of Sound Sources Using Icosahedral CNNs
• Signal-Aware Direction-of-Arrival Estimation Using Attention Mechanisms
• Analytical model for the relation between signal bandwidth and spatial resolution in Steered-Response Power Phase Transform (SRP-PHAT) maps
• wav2pos: Sound Source Localization using Masked Autoencoders
• Scalable-Complexity Steered Response Power Mapping based on Low-Rank and Sparse Interpolation
• Leveraging Sound Source Trajectories for Universal Sound Separation
• CNN-based Robust Sound Source Localization with SRP-PHAT for the Extreme Edge
• The trajectoRIR Database: Room Acoustic Recordings Along a Trajectory of Moving Microphones