Symbolic Music Generation with Diffusion Models

Abstract

Score-based generative models and diffusion probabilistic models have been successful at generating high-quality samples in continuous domains such as images and audio. However, due to their Langevin-inspired sampling mechanisms, their application to discrete and sequential data has been limited. In this work, we present a technique for training diffusion models on sequential data by parameterizing the discrete domain in the continuous latent space of a pre-trained variational autoencoder. Our method is non-autoregressive and learns to generate sequences of latent embeddings through the reverse process and offers parallel generation with a constant number of iterative refinement steps. We apply this technique to modeling symbolic music and show strong unconditional generation and post-hoc conditional infilling results compared to autoregressive language models operating over the same continuous embeddings.

arXiv commentISMIR 2021

References in corpus (15)

• Auto-Encoding Variational Bayes
• GANs Trained by a Two Time-Scale Update Rule Converge to a Local Nash Equilibrium
• Generative Modeling by Estimating Gradients of the Data Distribution
• Neural Audio Synthesis of Musical Notes with WaveNet Autoencoders
• A Hierarchical Latent Vector Model for Learning Long-Term Structure in Music
• GANSynth: Adversarial Neural Audio Synthesis
• Enabling Factorized Piano Music Modeling and Generation with the MAESTRO Dataset
• Improved Techniques for Training Score-Based Generative Models
• Jukebox: A Generative Model for Music
• Implicit Generation and Generalization in Energy-Based Models
• Counterpoint by Convolution
• DDSP: Differentiable Digital Signal Processing
• Imposing higher-level Structure in Polyphonic Music Generation using Convolutional Restricted Boltzmann Machines and Constraints
• Learning a Latent Space of Multitrack Measures
• PIANOTREE VAE: Structured Representation Learning for Polyphonic Music

Cited by in corpus (6)

• Structured Denoising Diffusion Models in Discrete State-Spaces
• CSDI: Conditional Score-based Diffusion Models for Probabilistic Time Series Imputation
• On Fast Sampling of Diffusion Probabilistic Models
• Gotta Go Fast When Generating Data with Score-Based Models
• Score-based Generative Modeling in Latent Space
• D2C: Diffusion-Denoising Models for Few-shot Conditional Generation