Learning Music Sequence Representation from Text Supervision
arXiv:2305.19602 · doi:10.1109/ICASSP43922.2022.9746131
Abstract
Music representation learning is notoriously difficult for its complex human-related concepts contained in the sequence of numerical signals. To excavate better MUsic SEquence Representation from labeled audio, we propose a novel text-supervision pre-training method, namely MUSER. MUSER adopts an audio-spectrum-text tri-modal contrastive learning framework, where the text input could be any form of meta-data with the help of text templates while the spectrum is derived from an audio sequence. Our experiments reveal that MUSER could be more flexibly adapted to downstream tasks compared with the current data-hungry pre-training method, and it only requires 0.056% of pre-training data to achieve the state-of-the-art performance.
References in corpus (8)
- Automatic tagging using deep convolutional neural networks
- End-to-end learning for music audio tagging at scale
- musicnn: Pre-trained convolutional neural networks for music audio tagging
- Toward Interpretable Music Tagging with Self-Attention
- Contrastive Learning of Musical Representations
- Codified audio language modeling learns useful representations for music information retrieval
- A Simple Long-Tailed Recognition Baseline via Vision-Language Model
- Representation Learning of Music Using Artist, Album, and Track Information