A Universal Music Translation Network
arXiv:1805.07848
Abstract
We present a method for translating music across musical instruments, genres, and styles. This method is based on a multi-domain wavenet autoencoder, with a shared encoder and a disentangled latent space that is trained end-to-end on waveforms. Employing a diverse training dataset and large net capacity, the domain-independent encoder allows us to translate even from musical domains that were not seen during training. The method is unsupervised and does not rely on supervision in the form of matched samples between domains or musical transcriptions. We evaluate our method on NSynth, as well as on a dataset collected from professional musicians, and achieve convincing translations, even when translating from whistling, potentially enabling the creation of instrumental music by untrained humans.
References in corpus (5)
Cited by in corpus (14)
- Self-Supervised VQ-VAE for One-Shot Music Style Transfer
- Unsupervised Multimodal Representation Learning across Medical Images and Reports
- TimbreTron: A WaveNet(CycleGAN(CQT(Audio))) Pipeline for Musical Timbre Transfer
- Modulated Variational auto-Encoders for many-to-many musical timbre transfer
- Latent Translation: Crossing Modalities by Bridging Generative Models
- Fork or Fail: Cycle-Consistent Training with Many-to-One Mappings
- Encoding Musical Style with Transformer Autoencoders
- Learning Disentangled Representations of Timbre and Pitch for Musical Instrument Sounds Using Gaussian Mixture Variational Autoencoders
- Play as You Like: Timbre-enhanced Multi-modal Music Style Transfer
- Speech-to-Singing Conversion in an Encoder-Decoder Framework
- Speech-to-Singing Conversion based on Boundary Equilibrium GAN
- Musical Speech: A Transformer-based Composition Tool
- End-To-End Dilated Variational Autoencoder with Bottleneck Discriminative Loss for Sound Morphing -- A Preliminary Study
- Adversarially Trained Autoencoders for Parallel-Data-Free Voice Conversion