Polyphonic Sound Event Detection by using Capsule Neural Networks

Abstract

Artificial sound event detection (SED) has the aim to mimic the human ability to perceive and understand what is happening in the surroundings. Nowadays, Deep Learning offers valuable techniques for this goal such as Convolutional Neural Networks (CNNs). The Capsule Neural Network (CapsNet) architecture has been recently introduced in the image processing field with the intent to overcome some of the known limitations of CNNs, specifically regarding the scarce robustness to affine transformations (i.e., perspective, size, orientation) and the detection of overlapped images. This motivated the authors to employ CapsNets to deal with the polyphonic-SED task, in which multiple sound events occur simultaneously. Specifically, we propose to exploit the capsule units to represent a set of distinctive properties for each individual sound event. Capsule units are connected through a so-called "dynamic routing" that encourages learning part-whole relationships and improves the detection performance in a polyphonic context. This paper reports extensive evaluations carried out on three publicly available datasets, showing how the CapsNet-based algorithm not only outperforms standard CNNs but also allows to achieve the best results with respect to the state of the art algorithms.

References in corpus (5)

• ADADELTA: An Adaptive Learning Rate Method
• Deep Convolutional Neural Networks and Data Augmentation for Environmental Sound Classification
• Convolutional Recurrent Neural Networks for Polyphonic Sound Event Detection
• A report on sound event detection with different binaural features
• Audio Surveillance: a Systematic Review

Cited by in corpus (8)

• Artificial Intelligence in the Creative Industries: A Review
• You Only Hear Once: A YOLO-like Algorithm for Audio Segmentation and Sound Event Detection
• Novel Dual-Channel Long Short-Term Memory Compressed Capsule Networks for Emotion Recognition
• A Four-Stage Data Augmentation Approach to ResNet-Conformer Based Acoustic Modeling for Sound Event Localization and Detection
• Cross-Referencing Self-Training Network for Sound Event Detection in Audio Mixtures
• Detecting Sound-Absorbing Materials in a Room from a Single Impulse Response using a CRNN
• Sequential Routing Framework: Fully Capsule Network-based Speech Recognition
• Soft-Median Choice: An Automatic Feature Smoothing Method for Sound Event Detection