Audio Event-Relational Graph Representation Learning for Acoustic Scene Classification
arXiv:2310.03889 · doi:10.1109/LSP.2023.3319233
Abstract
Most deep learning-based acoustic scene classification (ASC) approaches identify scenes based on acoustic features converted from audio clips containing mixed information entangled by polyphonic audio events (AEs). However, these approaches have difficulties in explaining what cues they use to identify scenes. This paper conducts the first study on disclosing the relationship between real-life acoustic scenes and semantic embeddings from the most relevant AEs. Specifically, we propose an event-relational graph representation learning (ERGL) framework for ASC to classify scenes, and simultaneously answer clearly and straightly which cues are used in classifying. In the event-relational graph, embeddings of each event are treated as nodes, while relationship cues derived from each pair of nodes are described by multi-dimensional edge features. Experiments on a real-life ASC dataset show that the proposed ERGL achieves competitive performance on ASC by learning embeddings of only a limited number of AEs. The results show the feasibility of recognizing diverse acoustic scenes based on the audio event-relational graph. Visualizations of graph representations learned by ERGL are available here (https://github.com/Yuanbo2020/ERGL).
IEEE Signal Processing Letters, doi: 10.1109/LSP.2023.3319233
References in corpus (8)
- A Comprehensive Survey on Graph Neural Networks
- Decoupled Weight Decay Regularization
- Acoustic Scene Classification
- Nonparametric regression using deep neural networks with ReLU activation function
- Learning Multi-dimensional Edge Feature-based AU Relation Graph for Facial Action Unit Recognition
- Joint Analysis of Sound Events and Acoustic Scenes Using Multitask Learning
- Audio-visual scene classification: analysis of DCASE 2021 Challenge submissions
- Learning Graph Representation of Person-specific Cognitive Processes from Audio-visual Behaviours for Automatic Personality Recognition