Unsupervised Text Generation by Learning from Search
arXiv:2007.08557
Abstract
In this work, we present TGLS, a novel framework to unsupervised Text Generation by Learning from Search. We start by applying a strong search algorithm (in particular, simulated annealing) towards a heuristically defined objective that (roughly) estimates the quality of sentences. Then, a conditional generative model learns from the search results, and meanwhile smooth out the noise of search. The alternation between search and learning can be repeated for performance bootstrapping. We demonstrate the effectiveness of TGLS on two real-world natural language generation tasks, paraphrase generation and text formalization. Our model significantly outperforms unsupervised baseline methods in both tasks. Especially, it achieves comparable performance with the state-of-the-art supervised methods in paraphrase generation.
References in corpus (8)
- A Deep Generative Framework for Paraphrase Generation
- Style Transfer as Unsupervised Machine Translation
- A Dual Reinforcement Learning Framework for Unsupervised Text Style Transfer
- On Variational Learning of Controllable Representations for Text without Supervision
- Paraphrase Generation with Latent Bag of Words
- Formality Style Transfer with Hybrid Textual Annotations
- Paraphrase Generation with Deep Reinforcement Learning
- Zero-Shot Paraphrase Generation with Multilingual Language Models