Convolutional Recurrent Neural Networks for Polyphonic Sound Event Detection

Abstract

Sound events often occur in unstructured environments where they exhibit wide variations in their frequency content and temporal structure. Convolutional neural networks (CNN) are able to extract higher level features that are invariant to local spectral and temporal variations. Recurrent neural networks (RNNs) are powerful in learning the longer term temporal context in the audio signals. CNNs and RNNs as classifiers have recently shown improved performances over established methods in various sound recognition tasks. We combine these two approaches in a Convolutional Recurrent Neural Network (CRNN) and apply it on a polyphonic sound event detection task. We compare the performance of the proposed CRNN method with CNN, RNN, and other established methods, and observe a considerable improvement for four different datasets consisting of everyday sound events.

arXiv commentAccepted for IEEE Transactions on Audio, Speech and Language Processing, Special Issue on Sound Scene and Event Analysis

References in corpus (9)

• Adam: A Method for Stochastic Optimization
• Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift
• Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translation
• How transferable are features in deep neural networks?
• Deep Speech 2: End-to-End Speech Recognition in English and Mandarin
• A Theoretically Grounded Application of Dropout in Recurrent Neural Networks
• Recurrent Neural Networks for Polyphonic Sound Event Detection in Real Life Recordings
• Convolutional Recurrent Neural Networks for Music Classification
• Acoustic event detection for multiple overlapping similar sources

Cited by in corpus (78)

• Machine learning in acoustics: theory and applications
• Sound Event Localization and Detection of Overlapping Sources Using Convolutional Recurrent Neural Networks
• Computational bioacoustics with deep learning: a review and roadmap
• Automatic acoustic detection of birds through deep learning: the first Bird Audio Detection challenge
• Sound Event Detection: A Tutorial
• Deep Neural Network for Respiratory Sound Classification in Wearable Devices Enabled by Patient Specific Model Tuning
• Polyphonic Sound Event Detection and Localization using a Two-Stage Strategy
• SELD-TCN: Sound Event Localization & Detection via Temporal Convolutional Networks
• Quaternion Convolutional Neural Networks for Detection and Localization of 3D Sound Events
• A report on sound event detection with different binaural features
• Polyphonic Sound Event Detection by using Capsule Neural Networks
• You Only Hear Once: A YOLO-like Algorithm for Audio Segmentation and Sound Event Detection
• Multiscale CNN based Deep Metric Learning for Bioacoustic Classification: Overcoming Training Data Scarcity Using Dynamic Triplet Loss
• A Sequence Matching Network for Polyphonic Sound Event Localization and Detection
• Sound event detection using weakly labeled dataset with stacked convolutional and recurrent neural network
• Cross-task learning for audio tagging, sound event detection and spatial localization: DCASE 2019 baseline systems
• DCASE 2018 Challenge Surrey Cross-Task convolutional neural network baseline
• Duration robust weakly supervised sound event detection
• A Four-Stage Data Augmentation Approach to ResNet-Conformer Based Acoustic Modeling for Sound Event Localization and Detection
• Finding Strength in Weakness: Learning to Separate Sounds with Weak Supervision
• On Multitask Loss Function for Audio Event Detection and Localization
• Cross-Referencing Self-Training Network for Sound Event Detection in Audio Mixtures
• Joint Analysis of Sound Events and Acoustic Scenes Using Multitask Learning
• Multimodal Urban Sound Tagging with Spatiotemporal Context
• Large-scale weakly supervised audio classification using gated convolutional neural network
• Sound Event Detection Transformer: An Event-based End-to-End Model for Sound Event Detection
• Improving performance and inference on audio classification tasks using capsule networks
• Enemy Spotted: in-game gun sound dataset for gunshot classification and localization
• A Multi-grained based Attention Network for Semi-supervised Sound Event Detection
• Modeling Long-Range Context for Concurrent Dialogue Acts Recognition
• Enabling Early Audio Event Detection with Neural Networks
• WaveTransformer: A Novel Architecture for Audio Captioning Based on Learning Temporal and Time-Frequency Information
• Proposal-based Few-shot Sound Event Detection for Speech and Environmental Sounds with Perceivers
• On the performance of residual block design alternatives in convolutional neural networks for end-to-end audio classification
• An Attention Mechanism for Musical Instrument Recognition
• DeFT-Mamba: Universal Multichannel Sound Separation and Polyphonic Audio Classification
• Stutter Diagnosis and Therapy System Based on Deep Learning
• A General Network Architecture for Sound Event Localization and Detection Using Transfer Learning and Recurrent Neural Network
• Acoustic Event Detection with Classifier Chains
• Increase Apparent Public Speaking Fluency By Speech Augmentation
• Sound Event Detection Utilizing Graph Laplacian Regularization with Event Co-occurrence
• Unsupervised Learning of Semantic Audio Representations
• Sound event detection via dilated convolutional recurrent neural networks
• Sound Event Detection of Weakly Labelled Data with CNN-Transformer and Automatic Threshold Optimization
• Multi-Task Learning for Interpretable Weakly Labelled Sound Event Detection
• Streaming ResLSTM with Causal Mean Aggregation for Device-Directed Utterance Detection
• VOICe: A Sound Event Detection Dataset For Generalizable Domain Adaptation
• Specialized Decision Surface and Disentangled Feature for Weakly-Supervised Polyphonic Sound Event Detection
• Weakly Labeled Sound Event Detection Using Tri-training and Adversarial Learning
• An Ensemble Framework of Voice-Based Emotion Recognition System for Films and TV Programs
• A two-step system for sound event localization and detection
• Driving Style Representation in Convolutional Recurrent Neural Network Model of Driver Identification
• Sound Event Localization and Detection using Squeeze-Excitation Residual CNNs
• Sound Event Detection by Multitask Learning of Sound Events and Scenes with Soft Scene Labels
• Sound Event Detection Using Graph Laplacian Regularization Based on Event Co-occurrence
• A Capsule based Approach for Polyphonic Sound Event Detection
• Improving Polyphonic Sound Event Detection on Multichannel Recordings with the Sørensen-Dice Coefficient Loss and Transfer Learning
• Non-Negative Matrix Factorization-Convolutional Neural Network (NMF-CNN) For Sound Event Detection
• Joint Analysis of Acoustic Events and Scenes Based on Multitask Learning
• Evaluating robustness of You Only Hear Once(YOHO) Algorithm on noisy audios in the VOICe Dataset
• Guided learning for weakly-labeled semi-supervised sound event detection
• Sound Event Detection Using Duration Robust Loss Function
• Active Learning for Sound Event Detection
• Cross-scale Attention Model for Acoustic Event Classification
• Sound Event Detection Based on Curriculum Learning Considering Learning Difficulty of Events
• Impact of Sound Duration and Inactive Frames on Sound Event Detection Performance
• Semi-supervised Acoustic Event Detection based on tri-training
• Adaptive Convolution for Semantic Role Labeling
• Towards Open World Sound Event Detection
• Convolutional Recurrent Neural Networks for Bird Audio Detection
• Objective Human Affective Vocal Expression Detection and Automatic Classification with Stochastic Models and Learning Systems
• Conditioned Time-Dilated Convolutions for Sound Event Detection
• Sound Event Detection with Adaptive Frequency Selection
• Self-supervised Attention Model for Weakly Labeled Audio Event Classification
• Interactive Lungs Auscultation with Reinforcement Learning Agent
• Towards Citizen Science for Smart Cities: A Framework for a Collaborative Game of Bird Call Recognition Based on Internet of Sound Practices
• A Comparative Study of Speaker Role Identification in Air Traffic Communication Using Deep Learning Approaches
• Adaptive Multi-scale Detection of Acoustic Events