Generating lyrics with variational autoencoder and multi-modal artist embeddings
arXiv:1812.08318
Abstract
We present a system for generating song lyrics lines conditioned on the style of a specified artist. The system uses a variational autoencoder with artist embeddings. We propose the pre-training of artist embeddings with the representations learned by a CNN classifier, which is trained to predict artists based on MEL spectrograms of their song clips. This work is the first step towards combining audio and text modalities of songs for generating lyrics conditioned on the artist's style. Our preliminary results suggest that there is a benefit in initializing artists' embeddings with the representations learned by a spectrogram classifier.
5 pages, 5 tables, 1 figure
References in corpus (5)
- Very Deep Convolutional Networks for Large-Scale Image Recognition
- Convolutional Neural Networks for Sentence Classification
- Toward Controlled Generation of Text
- Image Inspired Poetry Generation in XiaoIce
- JTAV: Jointly Learning Social Media Content Representation by Fusing Textual, Acoustic, and Visual Features
Cited by in corpus (6)
- Style Transfer for Texts: Retrain, Report Errors, Compare with Rewrites
- Decomposing Textual Information For Style Transfer
- Introducing Aspects of Creativity in Automatic Poetry Generation
- Generation of lyrics lines conditioned on music audio clips
- LyricJam: A system for generating lyrics for live instrumental music
- Stylized Text Generation Using Wasserstein Autoencoders with a Mixture of Gaussian Prior