Improving the Diversity of Unsupervised Paraphrasing with Embedding Outputs

Abstract

We present a novel technique for zero-shot paraphrase generation. The key contribution is an end-to-end multilingual paraphrasing model that is trained using translated parallel corpora to generate paraphrases into "meaning spaces" -- replacing the final softmax layer with word embeddings. This architectural modification, plus a training procedure that incorporates an autoencoding objective, enables effective parameter sharing across languages for more fluent monolingual rewriting, and facilitates fluency and diversity in generation. Our continuous-output paraphrase generation models outperform zero-shot paraphrasing baselines when evaluated on two languages using a battery of computational metrics as well as in human assessment.

References in corpus (5)

• Neural Paraphrase Generation with Stacked Residual LSTM Networks
• A Deep Generative Framework for Paraphrase Generation
• Von Mises-Fisher Loss for Training Sequence to Sequence Models with Continuous Outputs
• Learning Semantic Sentence Embeddings using Sequential Pair-wise Discriminator
• Zero-Shot Paraphrase Generation with Multilingual Language Models