Paraphrase Generation as Unsupervised Machine Translation

Abstract

In this paper, we propose a new paradigm for paraphrase generation by treating the task as unsupervised machine translation (UMT) based on the assumption that there must be pairs of sentences expressing the same meaning in a large-scale unlabeled monolingual corpus. The proposed paradigm first splits a large unlabeled corpus into multiple clusters, and trains multiple UMT models using pairs of these clusters. Then based on the paraphrase pairs produced by these UMT models, a unified surrogate model can be trained to serve as the final \sts model to generate paraphrases, which can be directly used for test in the unsupervised setup, or be finetuned on labeled datasets in the supervised setup. The proposed method offers merits over machine-translation-based paraphrase generation methods, as it avoids reliance on bilingual sentence pairs. It also allows human intervene with the model so that more diverse paraphrases can be generated using different filtering criteria. Extensive experiments on existing paraphrase dataset for both the supervised and unsupervised setups demonstrate the effectiveness the proposed paradigm.

To appear at COLING 2022

References in corpus (32)

• Adam: A Method for Stochastic Optimization
• Neural Machine Translation by Jointly Learning to Align and Translate
• Sequence to Sequence Learning with Neural Networks
• Cross-lingual Language Model Pretraining
• Sequence Level Training with Recurrent Neural Networks
• Style Transfer from Non-Parallel Text by Cross-Alignment
• Multilingual Denoising Pre-training for Neural Machine Translation
• MASS: Masked Sequence to Sequence Pre-training for Language Generation
• Deep Reinforcement Learning for Dialogue Generation
• Unsupervised Statistical Machine Translation
• Phrase-Based & Neural Unsupervised Machine Translation
• A Simple, Fast Diverse Decoding Algorithm for Neural Generation
• Neural Paraphrase Generation with Stacked Residual LSTM Networks
• Unsupervised Machine Translation Using Monolingual Corpora Only
• A Deep Generative Framework for Paraphrase Generation
• An Effective Approach to Unsupervised Machine Translation
• Lagging Inference Networks and Posterior Collapse in Variational Autoencoders
• Unsupervised Neural Machine Translation
• Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks
• Joint Copying and Restricted Generation for Paraphrase
• Unsupervised Paraphrase Generation using Pre-trained Language Models
• Delete, Retrieve, Generate: A Simple Approach to Sentiment and Style Transfer
• Paraphrase Generation with Latent Bag of Words
• Unsupervised Neural Machine Translation Initialized by Unsupervised Statistical Machine Translation
• D-PAGE: Diverse Paraphrase Generation
• Learning to Paraphrase for Question Answering
• A task in a suit and a tie: paraphrase generation with semantic augmentation
• Neural Machine Translation For Paraphrase Generation
• ChineseBERT: Chinese Pretraining Enhanced by Glyph and Pinyin Information
• Unsupervised Paraphrasing by Simulated Annealing
• Controllable Paraphrase Generation with a Syntactic Exemplar
• Reference Language based Unsupervised Neural Machine Translation