Plug and Play Autoencoders for Conditional Text Generation
arXiv:2010.02983
Abstract
Text autoencoders are commonly used for conditional generation tasks such as style transfer. We propose methods which are plug and play, where any pretrained autoencoder can be used, and only require learning a mapping within the autoencoder's embedding space, training embedding-to-embedding (Emb2Emb). This reduces the need for labeled training data for the task and makes the training procedure more efficient. Crucial to the success of this method is a loss term for keeping the mapped embedding on the manifold of the autoencoder and a mapping which is trained to navigate the manifold by learning offset vectors. Evaluations on style transfer tasks both with and without sequence-to-sequence supervision show that our method performs better than or comparable to strong baselines while being up to four times faster.
arXiv commentTo be published in EMNLP 2020
References in corpus (10)
- Sequence to Sequence Learning with Neural Networks
- Style Transfer from Non-Parallel Text by Cross-Alignment
- MASS: Masked Sequence to Sequence Pre-training for Language Generation
- Adversarial Autoencoders
- Content preserving text generation with attribute controls
- Controllable Unsupervised Text Attribute Transfer via Editing Entangled Latent Representation
- Von Mises-Fisher Loss for Training Sequence to Sequence Models with Continuous Outputs
- Eval all, trust a few, do wrong to none: Comparing sentence generation models
- Domain Adaptive Text Style Transfer
- Simplifying Sentences with Sequence to Sequence Models