Exploiting Synchronized Lyrics And Vocal Features For Music Emotion Detection
arXiv:1901.04831
Abstract
One of the key points in music recommendation is authoring engaging playlists according to sentiment and emotions. While previous works were mostly based on audio for music discovery and playlists generation, we take advantage of our synchronized lyrics dataset to combine text representations and music features in a novel way; we therefore introduce the Synchronized Lyrics Emotion Dataset. Unlike other approaches that randomly exploited the audio samples and the whole text, our data is split according to the temporal information provided by the synchronization between lyrics and audio. This work shows a comparison between text-based and audio-based deep learning classification models using different techniques from Natural Language Processing and Music Information Retrieval domains. From the experiments on audio we conclude that using vocals only, instead of the whole audio data improves the overall performances of the audio classifier. In the lyrics experiments we exploit the state-of-the-art word representations applied to the main Deep Learning architectures available in literature. In our benchmarks the results show how the Bilinear LSTM classifier with Attention based on fastText word embedding performs better than the CNN applied on audio.
arXiv comment8 pages, 5 figures, 9 tables
References in corpus (10)
- Convolutional Neural Networks for Sentence Classification
- Character-Aware Neural Language Models
- Automatic tagging using deep convolutional neural networks
- Wave-U-Net: A Multi-Scale Neural Network for End-to-End Audio Source Separation
- A Tutorial on Deep Learning for Music Information Retrieval
- Kapre: On-GPU Audio Preprocessing Layers for a Quick Implementation of Deep Neural Network Models with Keras
- Music Mood Detection Based On Audio And Lyrics With Deep Neural Net
- Convolutional Attention Networks for Multimodal Emotion Recognition from Speech and Text Data
- Character Composition Model with Convolutional Neural Networks for Dependency Parsing on Morphologically Rich Languages
- A Multimodal Approach towards Emotion Recognition of Music using Audio and Lyrical Content