Adversarial Decomposition of Text Representation

Abstract

In this paper, we present a method for adversarial decomposition of text representation. This method can be used to decompose a representation of an input sentence into several independent vectors, each of them responsible for a specific aspect of the input sentence. We evaluate the proposed method on two case studies: the conversion between different social registers and diachronic language change. We show that the proposed method is capable of fine-grained controlled change of these aspects of the input sentence. It is also learning a continuous (rather than categorical) representation of the style of the sentence, which is more linguistically realistic. The model uses adversarial-motivational training and includes a special motivational loss, which acts opposite to the discriminator and encourages a better decomposition. Furthermore, we evaluate the obtained meaning embeddings on a downstream task of paraphrase detection and show that they significantly outperform the embeddings of a regular autoencoder.

arXiv commentAccepted at NAACL 2019

References in corpus (13)

• Sequence to Sequence Learning with Neural Networks
• Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modeling
• Fast and Accurate Deep Network Learning by Exponential Linear Units (ELUs)
• Universal Sentence Encoder
• Style Transfer from Non-Parallel Text by Cross-Alignment
• Pointer Sentinel Mixture Models
• An efficient framework for learning sentence representations
• Professor Forcing: A New Algorithm for Training Recurrent Networks
• Learning General Purpose Distributed Sentence Representations via Large Scale Multi-task Learning
• Context-aware Natural Language Generation with Recurrent Neural Networks
• Adversarially Regularized Autoencoders
• Generative Concatenative Nets Jointly Learn to Write and Classify Reviews
• Stopping GAN Violence: Generative Unadversarial Networks