Proposal-based Few-shot Sound Event Detection for Speech and Environmental Sounds with Perceivers
arXiv:2107.13616
Abstract
Many applications involve detecting and localizing specific sound events within long, untrimmed documents, including keyword spotting, medical observation, and bioacoustic monitoring for conservation. Deep learning techniques often set the state-of-the-art for these tasks. However, for some types of events, there is insufficient labeled data to train such models. In this paper, we propose a region proposal-based approach to few-shot sound event detection utilizing the Perceiver architecture. Motivated by a lack of suitable benchmark datasets, we generate two new few-shot sound event localization datasets: "Vox-CASE," using clips of celebrity speech as the sound event, and "ESC-CASE," using environmental sound events. Our highest performing proposed few-shot approaches achieve 0.483 and 0.418 F1-score, respectively, with 5-shot 5-way tasks on these two datasets. These represent relative improvements of 72.5% and 11.2% over strong proposal-free few-shot sound event detection baselines.
arXiv commentUpdated results based on additional experimentation and moved dataset generation prose to stand-alone section
References in corpus (11)
- Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks
- Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks
- Prototypical Networks for Few-shot Learning
- VoxCeleb2: Deep Speaker Recognition
- Convolutional Recurrent Neural Networks for Polyphonic Sound Event Detection
- Conformer: Convolution-augmented Transformer for Speech Recognition
- Perceiver: General Perception with Iterative Attention
- Few-Shot Keyword Spotting With Prototypical Networks
- Similarity R-C3D for Few-shot Temporal Activity Detection
- Revisiting Few-shot Activity Detection with Class Similarity Control
- A Study of Few-Shot Audio Classification