Polyphonic Sound Event Detection and Localization using a Two-Stage Strategy

Abstract

Sound event detection (SED) and localization refer to recognizing sound events and estimating their spatial and temporal locations. Using neural networks has become the prevailing method for SED. In the area of sound localization, which is usually performed by estimating the direction of arrival (DOA), learning-based methods have recently been developed. In this paper, it is experimentally shown that the trained SED model is able to contribute to the direction of arrival estimation (DOAE). However, joint training of SED and DOAE degrades the performance of both. Based on these results, a two-stage polyphonic sound event detection and localization method is proposed. The method learns SED first, after which the learned feature layers are transferred for DOAE. It then uses the SED ground truth as a mask to train DOAE. The proposed method is evaluated on the DCASE 2019 Task 3 dataset, which contains different overlapping sound events in different environments. Experimental results show that the proposed method is able to improve the performance of both SED and DOAE, and also performs significantly better than the baseline method.

arXiv comment6 pages, 2 figures, conference

References in corpus (12)

• Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift
• Deep Convolutional Neural Networks and Data Augmentation for Environmental Sound Classification
• Convolutional Recurrent Neural Networks for Polyphonic Sound Event Detection
• Sound Event Localization and Detection of Overlapping Sources Using Convolutional Recurrent Neural Networks
• Recurrent Neural Networks for Polyphonic Sound Event Detection in Real Life Recordings
• Broadband DOA estimation using Convolutional neural networks trained with noise signals
• Automatic tagging using deep convolutional neural networks
• Deep Neural Networks for Multiple Speaker Detection and Localization
• Exploiting Deep Neural Networks and Head Movements for Robust Binaural Localisation of Multiple Sources in Reverberant Environments
• General-purpose Tagging of Freesound Audio with AudioSet Labels: Task Description, Dataset, and Baseline
• Multi-Speaker Localization Using Convolutional Neural Network Trained with Noise
• Cross-task learning for audio tagging, sound event detection and spatial localization: DCASE 2019 baseline systems

Cited by in corpus (24)

• A Survey of Sound Source Localization with Deep Learning Methods
• Overview and Evaluation of Sound Event Localization and Detection in DCASE 2019
• Sound source detection, localization and classification using consecutive ensemble of CRNN models
• SALSA: Spatial Cue-Augmented Log-Spectrogram Features for Polyphonic Sound Event Localization and Detection
• A Sequence Matching Network for Polyphonic Sound Event Localization and Detection
• L3DAS22 Challenge: Learning 3D Audio Sources in a Real Office Environment
• A Dataset of Reverberant Spatial Sound Scenes with Moving Sources for Sound Event Localization and Detection
• SALSA-Lite: A Fast and Effective Feature for Polyphonic Sound Event Localization and Detection with Microphone Arrays
• A Four-Stage Data Augmentation Approach to ResNet-Conformer Based Acoustic Modeling for Sound Event Localization and Detection
• Event-Independent Network for Polyphonic Sound Event Localization and Detection
• A Dataset of Dynamic Reverberant Sound Scenes with Directional Interferers for Sound Event Localization and Detection
• Self-supervised Learning of Audio Representations from Audio-Visual Data using Spatial Alignment
• ACCDOA: Activity-Coupled Cartesian Direction of Arrival Representation for Sound Event Localization and Detection
• Sound Event Localization and Detection Using Activity-Coupled Cartesian DOA Vector and RD3net
• Leveraging Visual Supervision for Array-based Active Speaker Detection and Localization
• A General Network Architecture for Sound Event Localization and Detection Using Transfer Learning and Recurrent Neural Network
• Spatial mixup: Directional loudness modification as data augmentation for sound event localization and detection
• First Order Ambisonics Domain Spatial Augmentation for DNN-based Direction of Arrival Estimation
• L3DAS21 Challenge: Machine Learning for 3D Audio Signal Processing
• DcaseNet: An integrated pretrained deep neural network for detecting and classifying acoustic scenes and events
• Sound event localization and detection based on crnn using rectangular filters and channel rotation data augmentation
• What Makes Sound Event Localization and Detection Difficult? Insights from Error Analysis
• A two-step system for sound event localization and detection
• SLoClas: A Database for Joint Sound Localization and Classification