Audio Event Detection using Weakly Labeled Data

Abstract

Acoustic event detection is essential for content analysis and description of multimedia recordings. The majority of current literature on the topic learns the detectors through fully-supervised techniques employing strongly labeled data. However, the labels available for majority of multimedia data are generally weak and do not provide sufficient detail for such methods to be employed. In this paper we propose a framework for learning acoustic event detectors using only weakly labeled data. We first show that audio event detection using weak labels can be formulated as an Multiple Instance Learning problem. We then suggest two frameworks for solving multiple-instance learning, one based on support vector machines, and the other on neural networks. The proposed methods can help in removing the time consuming and expensive process of manually annotating data to facilitate fully supervised learning. Moreover, it can not only detect events in a recording but can also provide temporal locations of events in the recording. This helps in obtaining a complete description of the recording and is notable since temporal information was never known in the first place in weakly labeled data.

ACM Multimedia 2016

References in corpus (1)

• Deep Learning in Neural Networks: An Overview

Cited by in corpus (46)

• Machine learning in acoustics: theory and applications
• Neonatal seizure detection from raw multi-channel EEG using a fully convolutional architecture
• Unsupervised Feature Learning Based on Deep Models for Environmental Audio Tagging
• Weakly Labelled AudioSet Tagging with Attention Neural Networks
• Multi-level Attention Model for Weakly Supervised Audio Classification
• Sound event detection using weakly labeled dataset with stacked convolutional and recurrent neural network
• Surrey-cvssp system for DCASE2017 challenge task4
• DCASE 2018 Challenge Surrey Cross-Task convolutional neural network baseline
• Interpreting glottal flow dynamics for detecting COVID-19 from voice
• Experiments on the DCASE Challenge 2016: Acoustic Scene Classification and Sound Event Detection in Real Life Recording
• Deep CNN Framework for Audio Event Recognition using Weakly Labeled Web Data
• Finding Strength in Weakness: Learning to Separate Sounds with Weak Supervision
• Sound Event Detection with Sequentially Labelled Data Based on Connectionist Temporal Classification and Unsupervised Clustering
• Acoustic Scene Classification by Implicitly Identifying Distinct Sound Events
• DNN Transfer Learning based Non-linear Feature Extraction for Acoustic Event Classification
• Weakly Supervised Representation Learning for Unsynchronized Audio-Visual Events
• Large-scale weakly supervised audio classification using gated convolutional neural network
• SpeechCraft: A Fine-grained Expressive Speech Dataset with Natural Language Description
• Fully DNN-based Multi-label regression for audio tagging
• Advancing Multi-grained Alignment for Contrastive Language-Audio Pre-training
• Adaptive Multi-Class Audio Classification in Noisy In-Vehicle Environment
• Audio Set classification with attention model: A probabilistic perspective
• Secost: Sequential co-supervision for large scale weakly labeled audio event detection
• Weakly Supervised Deep Recurrent Neural Networks for Basic Dance Step Generation
• Attention and Localization based on a Deep Convolutional Recurrent Model for Weakly Supervised Audio Tagging
• An Attention Mechanism for Musical Instrument Recognition
• Learning Object Detection from Captions via Textual Scene Attributes
• Features and Kernels for Audio Event Recognition
• Adaptive pooling operators for weakly labeled sound event detection
• NIPS4Bplus: a richly annotated birdsong audio dataset
• Sound Event Detection of Weakly Labelled Data with CNN-Transformer and Automatic Threshold Optimization
• NELS -- Never-Ending Learner of Sounds
• Multi-Task Learning for Interpretable Weakly Labelled Sound Event Detection
• Weakly Labeled Sound Event Detection Using Tri-training and Adversarial Learning
• Learning Multi-instrument Classification with Partial Labels
• Audio Event and Scene Recognition: A Unified Approach using Strongly and Weakly Labeled Data
• SEEF-ALDR: A Speaker Embedding Enhancement Framework via Adversarial Learning based Disentangled Representation
• Deep Learning for Audio Transcription on Low-Resource Datasets
• Cosine-similarity penalty to discriminate sound classes in weakly-supervised sound event detection
• A joint separation-classification model for sound event detection of weakly labelled data
• Advancing Multiple Instance Learning with Attention Modeling for Categorical Speech Emotion Recognition
• Cross-scale Attention Model for Acoustic Event Classification
• mil-benchmarks: Standardized Evaluation of Deep Multiple-Instance Learning Techniques
• Multi-instance Domain Adaptation for Vaccine Adverse Event Detection
• A Joint Detection-Classification Model for Audio Tagging of Weakly Labelled Data
• Audio Tagging With Connectionist Temporal Classification Model Using Sequential Labelled Data