Semi-supervised source localization with deep generative modeling

Abstract

We propose a semi-supervised localization approach based on deep generative modeling with variational autoencoders (VAEs). Localization in reverberant environments remains a challenge, which machine learning (ML) has shown promise in addressing. Even with large data volumes, the number of labels available for supervised learning in reverberant environments is usually small. We address this issue by performing semi-supervised learning (SSL) with convolutional VAEs. The VAE is trained to generate the phase of relative transfer functions (RTFs), in parallel with a DOA classifier, on both labeled and unlabeled RTF samples. The VAE-SSL approach is compared with SRP-PHAT and fully-supervised CNNs. We find that VAE-SSL can outperform both SRP-PHAT and CNN in label-limited scenarios.

arXiv commentPublished in proceedings of IEEE International Workshop on Machine Learning for Signal Processing. arXiv admin note: substantial text overlap with arXiv:2101.10636

References in corpus (13)

• PyTorch: An Imperative Style, High-Performance Deep Learning Library
• Auto-Encoding Variational Bayes
• Variational Inference: A Review for Statisticians
• WaveNet: A Generative Model for Raw Audio
• An Introduction to Variational Autoencoders
• Semi-Supervised Learning with Deep Generative Models
• Deep Learning for Audio Signal Processing
• Machine learning in acoustics: theory and applications
• Pyro: Deep Universal Probabilistic Programming
• Pyroomacoustics: A Python package for audio room simulations and array processing algorithms
• Sound Event Localization and Detection of Overlapping Sources Using Convolutional Recurrent Neural Networks
• Multi-Speaker DOA Estimation Using Deep Convolutional Networks Trained with Noise Signals
• Spatial Source Subtraction Based on Incomplete Measurements of Relative Transfer Function

Cited by in corpus (5)

• A Survey of Sound Source Localization with Deep Learning Methods
• Audio scene monitoring using redundant ad-hoc microphone array networks
• A Review on Sound Source Localization in Robotics: Focusing on Deep Learning Methods
• Feature Selection based on Principal Component Analysis for Underwater Source Localization by Deep Learning
• SSLIDE: Sound Source Localization for Indoors based on Deep Learning