Modeling Melodic Feature Dependency with Modularized Variational Auto-Encoder
arXiv:1811.00162
Abstract
Automatic melody generation has been a long-time aspiration for both AI researchers and musicians. However, learning to generate euphonious melodies has turned out to be highly challenging. This paper introduces 1) a new variant of variational autoencoder (VAE), where the model structure is designed in a modularized manner in order to model polyphonic and dynamic music with domain knowledge, and 2) a hierarchical encoding/decoding strategy, which explicitly models the dependency between melodic features. The proposed framework is capable of generating distinct melodies that sounds natural, and the experiments for evaluating generated music clips show that the proposed model outperforms the baselines in human evaluation.
arXiv commentThe first three authors contributed equally
References in corpus (14)
- Auto-Encoding Variational Bayes
- Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modeling
- WaveNet: A Generative Model for Raw Audio
- Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translation
- Modeling Temporal Dependencies in High-Dimensional Sequences: Application to Polyphonic Music Generation and Transcription
- Neural Audio Synthesis of Musical Notes with WaveNet Autoencoders
- SampleRNN: An Unconditional End-to-End Neural Audio Generation Model
- A Hierarchical Latent Vector Model for Learning Long-Term Structure in Music
- MidiNet: A Convolutional Generative Adversarial Network for Symbolic-domain Music Generation
- Variational Recurrent Auto-Encoders
- DeepBach: a Steerable Model for Bach Chorales Generation
- Learning a Latent Space of Multitrack Measures
- Convolutional Generative Adversarial Networks with Binary Neurons for Polyphonic Music Generation
- BachProp: Learning to Compose Music in Multiple Styles