Encoding Musical Style with Transformer Autoencoders
arXiv:1912.05537
Abstract
We consider the problem of learning high-level controls over the global structure of generated sequences, particularly in the context of symbolic music generation with complex language models. In this work, we present the Transformer autoencoder, which aggregates encodings of the input data across time to obtain a global representation of style from a given performance. We show it is possible to combine this global representation with other temporally distributed embeddings, enabling improved control over the separate aspects of performance style and melody. Empirically, we demonstrate the effectiveness of our method on various music generation tasks on the MAESTRO dataset and a YouTube dataset with 10,000+ hours of piano performances, where we achieve improvements in terms of log-likelihood and mean listening scores as compared to baselines.
References in corpus (15)
- Sequence to Sequence Learning with Neural Networks
- WaveNet: A Generative Model for Raw Audio
- Conditional Image Generation with PixelCNN Decoders
- A Neural Representation of Sketch Drawings
- Neural Audio Synthesis of Musical Notes with WaveNet Autoencoders
- A Hierarchical Latent Vector Model for Learning Long-Term Structure in Music
- GANSynth: Adversarial Neural Audio Synthesis
- Enabling Factorized Piano Music Modeling and Generation with the MAESTRO Dataset
- A Universal Music Translation Network
- Learning to Groove with Inverse Sequence Transformations
- Few-shot Autoregressive Density Estimation: Towards Learning to Learn Distributions
- Discrete Autoencoders for Sequence Models
- Latent Constraints: Learning to Generate Conditionally from Unconditional Generative Models
- Exploring Conditioning for Generative Music Systems with Human-Interpretable Controls
- Improving Automatic Jazz Melody Generation by Transfer Learning Techniques
Cited by in corpus (6)
- A Comprehensive Survey on Deep Music Generation: Multi-level Representations, Algorithms, Evaluations, and Future Directions
- Multi-Instrumentalist Net: Unsupervised Generation of Music from Body Movements
- Automatic Composition of Guitar Tabs by Transformers and Groove Modeling
- Vector Quantized Contrastive Predictive Coding for Template-based Music Generation
- PopMAG: Pop Music Accompaniment Generation
- MuSLCAT: Multi-Scale Multi-Level Convolutional Attention Transformer for Discriminative Music Modeling on Raw Waveforms