RL-Duet: Online Music Accompaniment Generation Using Deep Reinforcement Learning

Abstract

This paper presents a deep reinforcement learning algorithm for online accompaniment generation, with potential for real-time interactive human-machine duet improvisation. Different from offline music generation and harmonization, online music accompaniment requires the algorithm to respond to human input and generate the machine counterpart in a sequential order. We cast this as a reinforcement learning problem, where the generation agent learns a policy to generate a musical note (action) based on previously generated context (state). The key of this algorithm is the well-functioning reward model. Instead of defining it using music composition rules, we learn this model from monophonic and polyphonic training data. This model considers the compatibility of the machine-generated note with both the machine-generated context and the human-generated context. Experiments show that this algorithm is able to respond to the human part and generate a melodic, harmonic and diverse machine part. Subjective evaluations on preferences show that the proposed algorithm generates music pieces of higher quality than the baseline method.

References in corpus (7)

• Objective-Reinforced Generative Adversarial Networks (ORGAN) for Sequence Generation Models
• C-RNN-GAN: Continuous recurrent neural networks with adversarial training
• GANSynth: Adversarial Neural Audio Synthesis
• MuseGAN: Multi-track Sequential Generative Adversarial Networks for Symbolic Music Generation and Accompaniment
• Counterpoint by Convolution
• The Bach Doodle: Approachable music composition with machine learning at scale
• Song From PI: A Musically Plausible Network for Pop Music Generation