Learning Interpretable Representation for Controllable Polyphonic Music Generation

Abstract

While deep generative models have become the leading methods for algorithmic composition, it remains a challenging problem to control the generation process because the latent variables of most deep-learning models lack good interpretability. Inspired by the content-style disentanglement idea, we design a novel architecture, under the VAE framework, that effectively learns two interpretable latent factors of polyphonic music: chord and texture. The current model focuses on learning 8-beat long piano composition segments. We show that such chord-texture disentanglement provides a controllable generation pathway leading to a wide spectrum of applications, including compositional style transfer, texture variation, and accompaniment arrangement. Both objective and subjective evaluations show that our method achieves a successful disentanglement and high quality controlled music generation.

References in corpus (13)

• Auto-Encoding Variational Bayes
• Challenging Common Assumptions in the Unsupervised Learning of Disentangled Representations
• A Hierarchical Latent Vector Model for Learning Long-Term Structure in Music
• MidiNet: A Convolutional Generative Adversarial Network for Symbolic-domain Music Generation
• Disentangling factors of variation in deep representations using adversarial training
• MuseGAN: Multi-track Sequential Generative Adversarial Networks for Symbolic Music Generation and Accompaniment
• MIDI-VAE: Modeling Dynamics and Instrumentation of Music with Applications to Style Transfer
• Imposing higher-level Structure in Polyphonic Music Generation using Convolutional Restricted Boltzmann Machines and Constraints
• Learning a Latent Space of Multitrack Measures
• Deep Music Analogy Via Latent Representation Disentanglement
• Music Style Transfer: A Position Paper
• From Artificial Neural Networks to Deep Learning for Music Generation -- History, Concepts and Trends
• Learning Complex Basis Functions for Invariant Representations of Audio

Cited by in corpus (4)

• The Ethical Implications of Generative Audio Models: A Systematic Literature Review
• Self-Supervised VQ-VAE for One-Shot Music Style Transfer
• Controllable deep melody generation via hierarchical music structure representation
• AccoMontage: Accompaniment Arrangement via Phrase Selection and Style Transfer