Exploiting Deep Neural Networks and Head Movements for Robust Binaural Localisation of Multiple Sources in Reverberant Environments

Abstract

This paper presents a novel machine-hearing system that exploits deep neural networks (DNNs) and head movements for robust binaural localisation of multiple sources in reverberant environments. DNNs are used to learn the relationship between the source azimuth and binaural cues, consisting of the complete cross-correlation function (CCF) and interaural level differences (ILDs). In contrast to many previous binaural hearing systems, the proposed approach is not restricted to localisation of sound sources in the frontal hemifield. Due to the similarity of binaural cues in the frontal and rear hemifields, front-back confusions often occur. To address this, a head movement strategy is incorporated in the localisation model to help reduce the front-back errors. The proposed DNN system is compared to a Gaussian mixture model (GMM) based system that employs interaural time differences (ITDs) and ILDs as localisation features. Our experiments show that the DNN is able to exploit information in the CCF that is not available in the ITD cue, which together with head movements substantially improves localisation accuracies under challenging acoustic scenarios in which multiple talkers and room reverberation are present.

arXiv comment10 pages

Cited by in corpus (14)

• Multi-Speaker DOA Estimation Using Deep Convolutional Networks Trained with Noise Signals
• The LOCATA Challenge: Acoustic Source Localization and Tracking
• Polyphonic Sound Event Detection and Localization using a Two-Stage Strategy
• Towards End-to-End Acoustic Localization using Deep Learning: from Audio Signal to Source Position Coordinates
• Robust Binaural Localization of a Target Sound Source by Combining Spectral Source Models and Deep Neural Networks
• Listening for Sirens: Locating and Classifying Acoustic Alarms in City Scenes
• The Cone of Silence: Speech Separation by Localization
• BeamLearning: an end-to-end Deep Learning approach for the angular localization of sound sources using raw multichannel acoustic pressure data
• Joining Sound Event Detection and Localization Through Spatial Segregation
• Signal-Aware Direction-of-Arrival Estimation Using Attention Mechanisms
• Multi-scale aggregation of phase information for reducing computational cost of CNN based DOA estimation
• Multi-Microphone Complex Spectral Mapping for Speech Dereverberation
• Ensemble of Discriminators for Domain Adaptation in Multiple Sound Source 2D Localization
• Study of speaker localization with binaural microphone array incorporating auditory filters and lateral angle estimation