Toward Controlled Generation of Text

Abstract

Generic generation and manipulation of text is challenging and has limited success compared to recent deep generative modeling in visual domain. This paper aims at generating plausible natural language sentences, whose attributes are dynamically controlled by learning disentangled latent representations with designated semantics. We propose a new neural generative model which combines variational auto-encoders and holistic attribute discriminators for effective imposition of semantic structures. With differentiable approximation to discrete text samples, explicit constraints on independent attribute controls, and efficient collaborative learning of generator and discriminators, our model learns highly interpretable representations from even only word annotations, and produces realistic sentences with desired attributes. Quantitative evaluation validates the accuracy of sentence and attribute generation.

Code adapted for text style transfer is released at: https://github.com/asyml/texar/tree/master/examples/text_style_transfer

References in corpus (13)

• Sequence to Sequence Learning with Neural Networks
• Conditional Image Synthesis With Auxiliary Classifier GANs
• InfoGAN: Interpretable Representation Learning by Information Maximizing Generative Adversarial Nets
• Deep Convolutional Inverse Graphics Network
• Training Deep Neural Networks on Noisy Labels with Bootstrapping
• Unsupervised Cross-Domain Image Generation
• GANS for Sequences of Discrete Elements with the Gumbel-softmax Distribution
• Harnessing Deep Neural Networks with Logic Rules
• Generative Visual Manipulation on the Natural Image Manifold
• Improved Variational Autoencoders for Text Modeling using Dilated Convolutions
• A Hybrid Convolutional Variational Autoencoder for Text Generation
• Texar: A Modularized, Versatile, and Extensible Toolkit for Text Generation
• Multi-space Variational Encoder-Decoders for Semi-supervised Labeled Sequence Transduction

Cited by in corpus (80)

• Recent Trends in Deep Learning Based Natural Language Processing
• Style Transfer from Non-Parallel Text by Cross-Alignment
• Recent Advances in Autoencoder-Based Representation Learning
• Learning to Generate Reviews and Discovering Sentiment
• A Survey of Knowledge-Enhanced Text Generation
• Hybrid Retrieval-Generation Reinforced Agent for Medical Image Report Generation
• Learning to Predict the Cosmological Structure Formation
• Unsupervised Machine Translation Using Monolingual Corpora Only
• Unsupervised Learning of Disentangled and Interpretable Representations from Sequential Data
• Deep Learning and Knowledge-Based Methods for Computer Aided Molecular Design -- Toward a Unified Approach: State-of-the-Art and Future Directions
• Emotional Chatting Machine: Emotional Conversation Generation with Internal and External Memory
• Style Transfer as Unsupervised Machine Translation
• Adversarial Generation of Natural Language
• Improved Variational Autoencoders for Text Modeling using Dilated Convolutions
• Deep Bayesian Inversion
• Recurrent Topic-Transition GAN for Visual Paragraph Generation
• Multi-Task Neural Models for Translating Between Styles Within and Across Languages
• Neural Text Generation: Past, Present and Beyond
• Multiple-Attribute Text Style Transfer
• Image Captioning at Will: A Versatile Scheme for Effectively Injecting Sentiments into Image Descriptions
• Unsupervised Text Style Transfer using Language Models as Discriminators
• Delete, Retrieve, Generate: A Simple Approach to Sentiment and Style Transfer
• Evaluating prose style transfer with the Bible
• Semi-Amortized Variational Autoencoders
• Z-Forcing: Training Stochastic Recurrent Networks
• A Tutorial on Deep Latent Variable Models of Natural Language
• Eval all, trust a few, do wrong to none: Comparing sentence generation models
• Spherical Latent Spaces for Stable Variational Autoencoders
• Controllable Abstractive Summarization
• Structured Generative Adversarial Networks
• Neural Language Generation: Formulation, Methods, and Evaluation
• Polyjuice: Generating Counterfactuals for Explaining, Evaluating, and Improving Models
• Unsupervised Discrete Sentence Representation Learning for Interpretable Neural Dialog Generation
• An Adversarial Approach to High-Quality, Sentiment-Controlled Neural Dialogue Generation
• Dear Sir or Madam, May I introduce the GYAFC Dataset: Corpus, Benchmarks and Metrics for Formality Style Transfer
• Variational Attention for Sequence-to-Sequence Models
• PepCVAE: Semi-Supervised Targeted Design of Antimicrobial Peptide Sequences
• Emotion-aware Chat Machine: Automatic Emotional Response Generation for Human-like Emotional Interaction
• MojiTalk: Generating Emotional Responses at Scale
• SALSA-TEXT : self attentive latent space based adversarial text generation
• Generating lyrics with variational autoencoder and multi-modal artist embeddings
• Dirichlet Variational Autoencoder for Text Modeling
• Conditional BERT Contextual Augmentation
• CDL: Curriculum Dual Learning for Emotion-Controllable Response Generation
• Fast Cross-domain Data Augmentation through Neural Sentence Editing
• Automatic Article Commenting: the Task and Dataset
• Incorporating Discriminator in Sentence Generation: a Gibbs Sampling Method
• Latent Space Factorisation and Manipulation via Matrix Subspace Projection
• Transformer-Based Neural Text Generation with Syntactic Guidance
• Language Style Transfer from Sentences with Arbitrary Unknown Styles
• Disentangled Representations for Manipulation of Sentiment in Text
• Knowledge Injection into Dialogue Generation via Language Models
• Can You be More Social? Injecting Politeness and Positivity into Task-Oriented Conversational Agents
• TreeGAN: Syntax-Aware Sequence Generation with Generative Adversarial Networks
• Composed Variational Natural Language Generation for Few-shot Intents
• Better Conversations by Modeling,Filtering,and Optimizing for Coherence and Diversity
• Extractive Summary as Discrete Latent Variables
• Decomposable Neural Paraphrase Generation
• Target Guided Emotion Aware Chat Machine
• Adversarial Connective-exploiting Networks for Implicit Discourse Relation Classification
• Explain Me the Painting: Multi-Topic Knowledgeable Art Description Generation
• Plug-and-Blend: A Framework for Controllable Story Generation with Blended Control Codes
• Style Transfer Through Multilingual and Feedback-Based Back-Translation
• Defo-Net: Learning Body Deformation using Generative Adversarial Networks
• A Semi-Supervised Framework for Misinformation Detection
• Disentanglement Analysis with Partial Information Decomposition
• Unsupervised Neural Dialect Translation with Commonality and Diversity Modeling
• Learning to Select Bi-Aspect Information for Document-Scale Text Content Manipulation
• Improvement in Machine Translation with Generative Adversarial Networks
• A Deep Network with Visual Text Composition Behavior
• Controlling Neural Machine Translation Formality with Synthetic Supervision
• Pseudo Siamese Network for Few-shot Intent Generation
• MT-VAE: Learning Motion Transformations to Generate Multimodal Human Dynamics
• MixPoet: Diverse Poetry Generation via Learning Controllable Mixed Latent Space
• Generating Contradictory, Neutral, and Entailing Sentences
• An Empirical Study of Extrapolation in Text Generation with Scalar Control
• Deep Generative Models: Deterministic Prediction with an Application in Inverse Rendering
• Efficient text generation of user-defined topic using generative adversarial networks
• Disentangling Semantics and Syntax in Sentence Embeddings with Pre-trained Language Models
• GTAE: Graph-Transformer based Auto-Encoders for Linguistic-Constrained Text Style Transfer