PIANOTREE VAE: Structured Representation Learning for Polyphonic Music

Abstract

The dominant approach for music representation learning involves the deep unsupervised model family variational autoencoder (VAE). However, most, if not all, viable attempts on this problem have largely been limited to monophonic music. Normally composed of richer modality and more complex musical structures, the polyphonic counterpart has yet to be addressed in the context of music representation learning. In this work, we propose the PianoTree VAE, a novel tree-structure extension upon VAE aiming to fit the polyphonic music learning. The experiments prove the validity of the PianoTree VAE via (i)-semantically meaningful latent code for polyphonic segments; (ii)-more satisfiable reconstruction aside of decent geometry learned in the latent space; (iii)-this model's benefits to the variety of the downstream music generation.

References in corpus (12)

• Auto-Encoding Variational Bayes
• A Recurrent Latent Variable Model for Sequential Data
• Modeling Temporal Dependencies in High-Dimensional Sequences: Application to Polyphonic Music Generation and Transcription
• C-RNN-GAN: Continuous recurrent neural networks with adversarial training
• A Hierarchical Latent Vector Model for Learning Long-Term Structure in Music
• MidiNet: A Convolutional Generative Adversarial Network for Symbolic-domain Music Generation
• MuseGAN: Multi-track Sequential Generative Adversarial Networks for Symbolic Music Generation and Accompaniment
• Counterpoint by Convolution
• Learning a Latent Space of Multitrack Measures
• Deep Music Analogy Via Latent Representation Disentanglement
• Continuous Melody Generation via Disentangled Short-Term Representations and Structural Conditions
• Inspecting and Interacting with Meaningful Music Representations using VAE

Cited by in corpus (7)

• Video Background Music Generation with Controllable Music Transformer
• A Comprehensive Survey on Deep Music Generation: Multi-level Representations, Algorithms, Evaluations, and Future Directions
• Structure-Enhanced Pop Music Generation via Harmony-Aware Learning
• Exploring Variational Auto-Encoder Architectures, Configurations, and Datasets for Generative Music Explainable AI
• A Domain-Knowledge-Inspired Music Embedding Space and a Novel Attention Mechanism for Symbolic Music Modeling
• Symbolic Music Generation with Diffusion Models
• AccoMontage: Accompaniment Arrangement via Phrase Selection and Style Transfer