Natural Language Processing Methods for Symbolic Music Generation and Information Retrieval: a Survey
arXiv:2402.17467 · doi:10.1145/3714457
Abstract
Several adaptations of Transformers models have been developed in various domains since its breakthrough in Natural Language Processing (NLP). This trend has spread into the field of Music Information Retrieval (MIR), including studies processing music data. However, the practice of leveraging NLP tools for symbolic music data is not novel in MIR. Music has been frequently compared to language, as they share several similarities, including sequential representations of text and music. These analogies are also reflected through similar tasks in MIR and NLP. This survey reviews NLP methods applied to symbolic music generation and information retrieval studies following two axes. We first propose an overview of representations of symbolic music adapted from natural language sequential representations. Such representations are designed by considering the specificities of symbolic music. These representations are then processed by models. Such models, possibly originally developed for text and adapted for symbolic music, are trained on various tasks. We describe these models, in particular deep learning models, through different prisms, highlighting music-specialized mechanisms. We finally present a discussion surrounding the effective use of NLP tools for symbolic music data. This includes technical issues regarding NLP methods and fundamental differences between text and music, which may open several doors for further research into more effectively adapting NLP tools to symbolic MIR.
36 pages, 5 figures, 4 tables
References in corpus (30)
- Llama 2: Open Foundation and Fine-Tuned Chat Models
- Evaluating Word Embedding Models: Methods and Experimental Results
- CCNet: Extracting High Quality Monolingual Datasets from Web Crawl Data
- A Functional Taxonomy of Music Generation Systems
- Between words and characters: A Brief History of Open-Vocabulary Modeling and Tokenization in NLP
- Video Background Music Generation with Controllable Music Transformer
- MMM : Exploring Conditional Multi-Track Music Generation with the Transformer
- PiRhDy: Learning Pitch-, Rhythm-, and Dynamics-aware Embeddings for Symbolic Music
- BERT-like Pre-training for Symbolic Piano Music Classification Tasks
- AI-Based Affective Music Generation Systems: A Review of Methods, and Challenges
- TeleMelody: Lyric-to-Melody Generation with a Template-Based Two-Stage Method
- Polyphonic Music Composition with LSTM Neural Networks and Reinforcement Learning
- ComposerX: Multi-Agent Symbolic Music Composition with LLMs
- Embeddings as representation for symbolic music
- Reconstructing Human Expressiveness in Piano Performances with a Transformer Network
- ChatMusician: Understanding and Generating Music Intrinsically with LLM
- MuseMorphose: Full-Song and Fine-Grained Piano Music Style Transfer with One Transformer VAE
- From Words to Music: A Study of Subword Tokenization Techniques in Symbolic Music Generation
- Machine Learning Techniques in Automatic Music Transcription: A Systematic Survey
- Can LLMs "Reason" in Music? An Evaluation of LLMs' Capability of Music Understanding and Generation
- Pitchclass2vec: Symbolic Music Structure Segmentation with Chord Embeddings
- Choir Transformer: Generating Polyphonic Music with Relative Attention on Transformer
- Generating Music with Structure Using Self-Similarity as Attention
- MuseBarControl: Enhancing Fine-Grained Control in Symbolic Music Generation through Pre-Training and Counterfactual Loss
- musicaiz: A Python Library for Symbolic Music Generation, Analysis and Visualization
- The Music Maestro or The Musically Challenged, A Massive Music Evaluation Benchmark for Large Language Models
- Symbolic music generation conditioned on continuous-valued emotions
- MelodyT5: A Unified Score-to-Score Transformer for Symbolic Music Processing
- Structured Multi-Track Accompaniment Arrangement via Style Prior Modelling
- An Comparative Analysis of Different Pitch and Metrical Grid Encoding Methods in the Task of Sequential Music Generation