AI-based soundscape analysis: Jointly identifying sound sources and predicting annoyance

Abstract

Soundscape studies typically attempt to capture the perception and understanding of sonic environments by surveying users. However, for long-term monitoring or assessing interventions, sound-signal-based approaches are required. To this end, most previous research focused on psycho-acoustic quantities or automatic sound recognition. Few attempts were made to include appraisal (e.g., in circumplex frameworks). This paper proposes an artificial intelligence (AI)-based dual-branch convolutional neural network with cross-attention-based fusion (DCNN-CaF) to analyze automatic soundscape characterization, including sound recognition and appraisal. Using the DeLTA dataset containing human-annotated sound source labels and perceived annoyance, the DCNN-CaF is proposed to perform sound source classification (SSC) and human-perceived annoyance rating prediction (ARP). Experimental findings indicate that (1) the proposed DCNN-CaF using loudness and Mel features outperforms the DCNN-CaF using only one of them. (2) The proposed DCNN-CaF with cross-attention fusion outperforms other typical AI-based models and soundscape-related traditional machine learning methods on the SSC and ARP tasks. (3) Correlation analysis reveals that the relationship between sound sources and annoyance is similar for humans and the proposed AI-based DCNN-CaF model. (4) Generalization tests show that the proposed model's ARP in the presence of model-unknown sound sources is consistent with expert expectations and can explain previous findings from the literature on sound-scape augmentation.

arXiv commentThe Journal of the Acoustical Society of America, 154 (5), 3145

References in corpus (7)

• Adam: A Method for Stochastic Optimization
• Acoustic Scene Classification
• Recurrent Neural Networks for Polyphonic Sound Event Detection in Real Life Recordings
• Overview and Evaluation of Sound Event Localization and Detection in DCASE 2019
• Sound event detection via dilated convolutional recurrent neural networks
• Relation-guided acoustic scene classification aided with event embeddings
• CT-SAT: Contextual Transformer for Sequential Audio Tagging

Cited by in corpus (2)

• Automating Urban Soundscape Enhancements with AI: In-situ Assessment of Quality and Restorativeness in Traffic-Exposed Residential Areas
• Definition-independent Formalization of Soundscapes: Towards a Formal Methodology