MidiNet: A Convolutional Generative Adversarial Network for Symbolic-domain Music Generation

Abstract

Most existing neural network models for music generation use recurrent neural networks. However, the recent WaveNet model proposed by DeepMind shows that convolutional neural networks (CNNs) can also generate realistic musical waveforms in the audio domain. Following this light, we investigate using CNNs for generating melody (a series of MIDI notes) one bar after another in the symbolic domain. In addition to the generator, we use a discriminator to learn the distributions of melodies, making it a generative adversarial network (GAN). Moreover, we propose a novel conditional mechanism to exploit available prior knowledge, so that the model can generate melodies either from scratch, by following a chord sequence, or by conditioning on the melody of previous bars (e.g. a priming melody), among other possibilities. The resulting model, named MidiNet, can be expanded to generate music with multiple MIDI channels (i.e. tracks). We conduct a user study to compare the melody of eight-bar long generated by MidiNet and by Google's MelodyRNN models, each time using the same priming melody. Result shows that MidiNet performs comparably with MelodyRNN models in being realistic and pleasant to listen to, yet MidiNet's melodies are reported to be much more interesting.

8 pages, Accepted to ISMIR (International Society of Music Information Retrieval) Conference 2017

References in corpus (11)

• Conditional Generative Adversarial Nets
• WaveNet: A Generative Model for Raw Audio
• NIPS 2016 Tutorial: Generative Adversarial Networks
• Modeling Temporal Dependencies in High-Dimensional Sequences: Application to Polyphonic Music Generation and Transcription
• Neural Audio Synthesis of Musical Notes with WaveNet Autoencoders
• C-RNN-GAN: Continuous recurrent neural networks with adversarial training
• SampleRNN: An Unconditional End-to-End Neural Audio Generation Model
• DeepBach: a Steerable Model for Bach Chorales Generation
• Convolutional Recurrent Neural Networks for Music Classification
• Fast Wavenet Generation Algorithm
• Composing Music with Grammar Argumented Neural Networks and Note-Level Encoding

Cited by in corpus (42)

• Ten Years of Generative Adversarial Nets (GANs): A survey of the state-of-the-art
• Scale- and Context-Aware Convolutional Non-intrusive Load Monitoring
• Video Background Music Generation with Controllable Music Transformer
• A Tutorial on Deep Learning for Music Information Retrieval
• MIDI-VAE: Modeling Dynamics and Instrumentation of Music with Applications to Style Transfer
• Physics-Informed Generative Adversarial Networks for Stochastic Differential Equations
• Machine Learning in NextG Networks via Generative Adversarial Networks
• Generative Adversarial Networks for Electronic Health Records: A Framework for Exploring and Evaluating Methods for Predicting Drug-Induced Laboratory Test Trajectories
• Learning a Latent Space of Multitrack Measures
• Convolutional Generative Adversarial Networks with Binary Neurons for Polyphonic Music Generation
• PIANOTREE VAE: Structured Representation Learning for Polyphonic Music
• Learning Interpretable Representation for Controllable Polyphonic Music Generation
• Score and Lyrics-Free Singing Voice Generation
• Learning Disentangled Representations for Timber and Pitch in Music Audio
• Relational Data Synthesis using Generative Adversarial Networks: A Design Space Exploration
• A Hierarchical Recurrent Neural Network for Symbolic Melody Generation
• Foley Music: Learning to Generate Music from Videos
• MIDI-Sandwich2: RNN-based Hierarchical Multi-modal Fusion Generation VAE networks for multi-track symbolic music generation
• Generative Melody Composition with Human-in-the-Loop Bayesian Optimization
• Adversarially Trained Multi-Singer Sequence-To-Sequence Singing Synthesizer
• Artificial Musical Intelligence: A Survey
• Inverse Estimation of Elastic Modulus Using Physics-Informed Generative Adversarial Networks
• The Beauty of Repetition in Machine Composition Scenarios
• PopMAG: Pop Music Accompaniment Generation
• CMTS: Conditional Multiple Trajectory Synthesizer for Generating Safety-critical Driving Scenarios
• Semi-Recurrent CNN-based VAE-GAN for Sequential Data Generation
• Statistical Parametric Speech Synthesis Using Generative Adversarial Networks Under A Multi-task Learning Framework
• SeismoGen: Seismic Waveform Synthesis Using Generative Adversarial Networks
• A Survey on Audio Synthesis and Audio-Visual Multimodal Processing
• Generating Music with a Self-Correcting Non-Chronological Autoregressive Model
• AccoMontage: Accompaniment Arrangement via Phrase Selection and Style Transfer
• Lead Sheet Generation and Arrangement by Conditional Generative Adversarial Network
• Neural Melody Composition from Lyrics
• Synthetic Epileptic Brain Activities Using Generative Adversarial Networks
• Enhanced Memory Network: The novel network structure for Symbolic Music Generation
• Symbolic Music Playing Techniques Generation as a Tagging Problem
• Synthetic Dynamic PMU Data Generation: A Generative Adversarial Network Approach
• Exploring Inherent Properties of the Monophonic Melody of Songs
• MIDI-Sandwich: Multi-model Multi-task Hierarchical Conditional VAE-GAN networks for Symbolic Single-track Music Generation
• Creation of Synthetic Networked PMU Data: A Generative Adversarial Network Approach
• Sinusoidal wave generating network based on adversarial learning and its application: synthesizing frog sounds for data augmentation
• Modeling Melodic Feature Dependency with Modularized Variational Auto-Encoder