Enriching Ontology with Temporal Commonsense for Low-Resource Audio Tagging

Abstract

Audio tagging aims at predicting sound events occurred in a recording. Traditional models require enormous laborious annotations, otherwise performance degeneration will be the norm. Therefore, we investigate robust audio tagging models in low-resource scenarios with the enhancement of knowledge graphs. Besides existing ontological knowledge, we further propose a semi-automatic approach that can construct temporal knowledge graphs on diverse domain-specific label sets. Moreover, we leverage a variant of relation-aware graph neural network, D-GCN, to combine the strength of the two knowledge types. Experiments on AudioSet and SONYC urban sound tagging datasets suggest the effectiveness of the introduced temporal knowledge, and the advantage of the combined KGs with D-GCN over single knowledge source.

arXiv commentCIKM 2021

References in corpus (4)

• YFCC100M: The New Data in Multimedia Research
• Scalable Multi-Hop Relational Reasoning for Knowledge-Aware Question Answering
• Addressing Missing Labels in Large-Scale Sound Event Recognition Using a Teacher-Student Framework With Loss Masking
• Language Generation with Multi-Hop Reasoning on Commonsense Knowledge Graph