Evaluation of CNN-based Automatic Music Tagging Models

Abstract

Recent advances in deep learning accelerated the development of content-based automatic music tagging systems. Music information retrieval (MIR) researchers proposed various architecture designs, mainly based on convolutional neural networks (CNNs), that achieve state-of-the-art results in this multi-label binary classification task. However, due to the differences in experimental setups followed by researchers, such as using different dataset splits and software versions for evaluation, it is difficult to compare the proposed architectures directly with each other. To facilitate further research, in this paper we conduct a consistent evaluation of different music tagging models on three datasets (MagnaTagATune, Million Song Dataset, and MTG-Jamendo) and provide reference results using common evaluation metrics (ROC-AUC and PR-AUC). Furthermore, all the models are evaluated with perturbed inputs to investigate the generalization capabilities concerning time stretch, pitch shift, dynamic range compression, and addition of white noise. For reproducibility, we provide the PyTorch implementations with the pre-trained models.

7 pages, 2 figures, Sound and Music Computing 2020 (SMC 2020)

References in corpus (2)

• Automatic tagging using deep convolutional neural networks
• Toward Interpretable Music Tagging with Self-Attention

Cited by in corpus (12)

• Enriched Music Representations with Multiple Cross-modal Contrastive Learning
• Codified audio language modeling learns useful representations for music information retrieval
• SpecTNT: a Time-Frequency Transformer for Music Audio
• audioLIME: Listenable Explanations Using Source Separation
• Multimodal Metric Learning for Tag-based Music Retrieval
• Mood Classification Using Listening Data
• Improving Sound Event Classification by Increasing Shift Invariance in Convolutional Neural Networks
• Emotion Embedding Spaces for Matching Music to Stories
• Deep Learning Based EDM Subgenre Classification using Mel-Spectrogram and Tempogram Features
• A Benchmarking Initiative for Audio-Domain Music Generation Using the Freesound Loop Dataset
• Unsupervised Source Separation By Steering Pretrained Music Models
• MuSLCAT: Multi-Scale Multi-Level Convolutional Attention Transformer for Discriminative Music Modeling on Raw Waveforms