SELD-TCN: Sound Event Localization & Detection via Temporal Convolutional Networks

Abstract

The understanding of the surrounding environment plays a critical role in autonomous robotic systems, such as self-driving cars. Extensive research has been carried out concerning visual perception. Yet, to obtain a more complete perception of the environment, autonomous systems of the future should also take acoustic information into account. Recent sound event localization and detection (SELD) frameworks utilize convolutional recurrent neural networks (CRNNs). However, considering the recurrent nature of CRNNs, it becomes challenging to implement them efficiently on embedded hardware. Not only are their computations strenuous to parallelize, but they also require high memory bandwidth and large memory buffers. In this work, we develop a more robust and hardware-friendly novel architecture based on a temporal convolutional network(TCN). The proposed framework (SELD-TCN) outperforms the state-of-the-art SELDnet performance on four different datasets. Moreover, SELD-TCN achieves 4x faster training time per epoch and 40x faster inference time on an ordinary graphics processing unit (GPU).

arXiv comment5 pages, 3 tables, 2 figures. Submitted to EUSIPCO 2020

References in corpus (8)

• Adam: A Method for Stochastic Optimization
• Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift
• An Empirical Evaluation of Generic Convolutional and Recurrent Networks for Sequence Modeling
• WaveNet: A Generative Model for Raw Audio
• Convolutional Recurrent Neural Networks for Polyphonic Sound Event Detection
• Recent Advances in Convolutional Neural Networks
• Broadband DOA estimation using Convolutional neural networks trained with noise signals
• Sound Event Detection in Multichannel Audio Using Spatial and Harmonic Features

Cited by in corpus (5)

• A Survey of Sound Source Localization with Deep Learning Methods
• L3DAS22 Challenge: Learning 3D Audio Sources in a Real Office Environment
• Online Handwriting Trajectory Reconstruction from Kinematic Sensors using Temporal Convolutional Network
• L3DAS21 Challenge: Machine Learning for 3D Audio Signal Processing
• Deep Multi-task Network for Delay Estimation and Echo Cancellation