Vector Quantized Contrastive Predictive Coding for Template-based Music Generation

Abstract

In this work, we propose a flexible method for generating variations of discrete sequences in which tokens can be grouped into basic units, like sentences in a text or bars in music. More precisely, given a template sequence, we aim at producing novel sequences sharing perceptible similarities with the original template without relying on any annotation; so our problem of generating variations is intimately linked to the problem of learning relevant high-level representations without supervision. Our contribution is two-fold: First, we propose a self-supervised encoding technique, named Vector Quantized Contrastive Predictive Coding which allows to learn a meaningful assignment of the basic units over a discrete set of codes, together with mechanisms allowing to control the information content of these learnt discrete representations. Secondly, we show how these compressed representations can be used to generate variations of a template sequence by using an appropriate attention pattern in the Transformer architecture. We illustrate our approach on the corpus of J.S. Bach chorales where we discuss the musical meaning of the learnt discrete codes and show that our proposed method allows to generate coherent and high-quality variations of a given template.

15 pages, 13 figures

References in corpus (18)

• A Simple Framework for Contrastive Learning of Visual Representations
• Neural Discrete Representation Learning
• The Curious Case of Neural Text Degeneration
• Data-Efficient Image Recognition with Contrastive Predictive Coding
• Generating Long Sequences with Sparse Transformers
• vq-wav2vec: Self-Supervised Learning of Discrete Speech Representations
• A Hierarchical Latent Vector Model for Learning Long-Term Structure in Music
• Generating Diverse High-Fidelity Images with VQ-VAE-2
• Music transcription modelling and composition using deep learning
• Large Memory Layers with Product Keys
• PixelSNAIL: An Improved Autoregressive Generative Model
• The Bach Doodle: Approachable music composition with machine learning at scale
• LakhNES: Improving multi-instrumental music generation with cross-domain pre-training
• Hierarchical Autoregressive Image Models with Auxiliary Decoders
• Latent Constraints: Learning to Generate Conditionally from Unconditional Generative Models
• Adaptively Sparse Transformers
• Encoding Musical Style with Transformer Autoencoders
• Sampling Variations of Lead Sheets

Cited by in corpus (7)

• A Comprehensive Survey on Deep Music Generation: Multi-level Representations, Algorithms, Evaluations, and Future Directions
• Variable-rate discrete representation learning
• The Piano Inpainting Application
• Spectrogram Inpainting for Interactive Generation of Instrument Sounds
• An Exploratory Study on Perceptual Spaces of the Singing Voice
• Incorporating Music Knowledge in Continual Dataset Augmentation for Music Generation
• Symbolic Music Loop Generation with VQ-VAE