Learning Semantic Sentence Embeddings using Sequential Pair-wise Discriminator

Abstract

In this paper, we propose a method for obtaining sentence-level embeddings. While the problem of securing word-level embeddings is very well studied, we propose a novel method for obtaining sentence-level embeddings. This is obtained by a simple method in the context of solving the paraphrase generation task. If we use a sequential encoder-decoder model for generating paraphrase, we would like the generated paraphrase to be semantically close to the original sentence. One way to ensure this is by adding constraints for true paraphrase embeddings to be close and unrelated paraphrase candidate sentence embeddings to be far. This is ensured by using a sequential pair-wise discriminator that shares weights with the encoder that is trained with a suitable loss function. Our loss function penalizes paraphrase sentence embedding distances from being too large. This loss is used in combination with a sequential encoder-decoder network. We also validated our method by evaluating the obtained embeddings for a sentiment analysis task. The proposed method results in semantic embeddings and outperforms the state-of-the-art on the paraphrase generation and sentiment analysis task on standard datasets. These results are also shown to be statistically significant.

COLING 2018 (accepted)

References in corpus (13)

• Efficient Estimation of Word Representations in Vector Space
• Distributed Representations of Words and Phrases and their Compositionality
• Sequence to Sequence Learning with Neural Networks
• Distributed Representations of Sentences and Documents
• Generative Adversarial Text to Image Synthesis
• A Neural Conversational Model
• word2vec Explained: deriving Mikolov et al.'s negative-sampling word-embedding method
• Neural Paraphrase Generation with Stacked Residual LSTM Networks
• Long Text Generation via Adversarial Training with Leaked Information
• A Deep Generative Framework for Paraphrase Generation
• Adversarial Feature Matching for Text Generation
• Learning to Ask: Neural Question Generation for Reading Comprehension
• Recurrent Topic-Transition GAN for Visual Paragraph Generation

Cited by in corpus (7)

• Improving Natural Language Processing Tasks with Human Gaze-Guided Neural Attention
• Attending to Discriminative Certainty for Domain Adaptation
• Granular Multimodal Attention Networks for Visual Dialog
• A Hybrid Natural Language Generation System Integrating Rules and Deep Learning Algorithms
• Dynamic Attention Networks for Task Oriented Grounding
• Improving the Diversity of Unsupervised Paraphrasing with Embedding Outputs
• Sentence-level Online Handwritten Chinese Character Recognition