Online Localization and Tracking of Multiple Moving Speakers in Reverberant Environments

Abstract

We address the problem of online localization and tracking of multiple moving speakers in reverberant environments. The paper has the following contributions. We use the direct-path relative transfer function (DP-RTF), an inter-channel feature that encodes acoustic information robust against reverberation, and we propose an online algorithm well suited for estimating DP-RTFs associated with moving audio sources. Another crucial ingredient of the proposed method is its ability to properly assign DP-RTFs to audio-source directions. Towards this goal, we adopt a maximum-likelihood formulation and we propose to use an exponentiated gradient (EG) to efficiently update source-direction estimates starting from their currently available values. The problem of multiple speaker tracking is computationally intractable because the number of possible associations between observed source directions and physical speakers grows exponentially with time. We adopt a Bayesian framework and we propose a variational approximation of the posterior filtering distribution associated with multiple speaker tracking, as well as an efficient variational expectation-maximization (VEM) solver. The proposed online localization and tracking method is thoroughly evaluated using two datasets that contain recordings performed in real environments.

arXiv commentIEEE Journal of Selected Topics in Signal Processing, 2019

References in corpus (6)

• Robust Localization and Tracking of Simultaneous Moving Sound Sources Using Beamforming and Particle Filtering
• EM Algorithms for Weighted-Data Clustering with Application to Audio-Visual Scene Analysis
• Audio-Visual Speaker Diarization Based on Spatiotemporal Bayesian Fusion
• An On-line Variational Bayesian Model for Multi-Person Tracking from Cluttered Scenes
• Multiple-Speaker Localization Based on Direct-Path Features and Likelihood Maximization with Spatial Sparsity Regularization
• Reverberant Sound Localization with a Robot Head Based on Direct-Path Relative Transfer Function

Cited by in corpus (7)

• The LOCATA Challenge: Acoustic Source Localization and Tracking
• Self-supervised Moving Vehicle Tracking with Stereo Sound
• Tracking Multiple Audio Sources with the von Mises Distribution and Variational EM
• Variational Bayesian Inference for Audio-Visual Tracking of Multiple Speakers
• Multichannel CRNN for Speaker Counting: an Analysis of Performance
• AcousticFusion: Fusing Sound Source Localization to Visual SLAM in Dynamic Environments
• A cascaded multiple-speaker localization and tracking system