Toward Interpretable Music Tagging with Self-Attention

Abstract

Self-attention is an attention mechanism that learns a representation by relating different positions in the sequence. The transformer, which is a sequence model solely based on self-attention, and its variants achieved state-of-the-art results in many natural language processing tasks. Since music composes its semantics based on the relations between components in sparse positions, adopting the self-attention mechanism to solve music information retrieval (MIR) problems can be beneficial. Hence, we propose a self-attention based deep sequence model for music tagging. The proposed architecture consists of shallow convolutional layers followed by stacked Transformer encoders. Compared to conventional approaches using fully convolutional or recurrent neural networks, our model is more interpretable while reporting competitive results. We validate the performance of our model with the MagnaTagATune and the Million Song Dataset. In addition, we demonstrate the interpretability of the proposed architecture with a heat map visualization.

13 pages, 12 figures; code: https://github.com/minzwon/self-attention-music-tagging

References in corpus (10)

• On the difficulty of training Recurrent Neural Networks
• WaveNet: A Generative Model for Raw Audio
• Self-Attention Generative Adversarial Networks
• A Structured Self-attentive Sentence Embedding
• Improving Generalization Performance by Switching from Adam to SGD
• Train longer, generalize better: closing the generalization gap in large batch training of neural networks
• Enabling Factorized Piano Music Modeling and Generation with the MAESTRO Dataset
• NSML: Meet the MLaaS platform with a real-world case study
• NSML: A Machine Learning Platform That Enables You to Focus on Your Models
• Explaining Deep Convolutional Neural Networks on Music Classification

Cited by in corpus (13)

• Evaluation of CNN-based Automatic Music Tagging Models
• SpecTNT: a Time-Frequency Transformer for Music Audio
• Metric Learning vs Classification for Disentangled Music Representation Learning
• Learning Music Sequence Representation from Text Supervision
• Tracing Back Music Emotion Predictions to Sound Sources and Intuitive Perceptual Qualities
• Sound Event Detection of Weakly Labelled Data with CNN-Transformer and Automatic Threshold Optimization
• Deep Learning Based EDM Subgenre Classification using Mel-Spectrogram and Tempogram Features
• Towards Musically Meaningful Explanations Using Source Separation
• ReconVAT: A Semi-Supervised Automatic Music Transcription Framework for Low-Resource Real-World Data
• Singer Identification Using Deep Timbre Feature Learning with KNN-Net
• Receptive-Field Regularized CNNs for Music Classification and Tagging
• Visualizing and Understanding Self-attention based Music Tagging
• MuSLCAT: Multi-Scale Multi-Level Convolutional Attention Transformer for Discriminative Music Modeling on Raw Waveforms