Adversarial Text Generation via Feature-Mover's Distance

Abstract

Generative adversarial networks (GANs) have achieved significant success in generating real-valued data. However, the discrete nature of text hinders the application of GAN to text-generation tasks. Instead of using the standard GAN objective, we propose to improve text-generation GAN via a novel approach inspired by optimal transport. Specifically, we consider matching the latent feature distributions of real and synthetic sentences using a novel metric, termed the feature-mover's distance (FMD). This formulation leads to a highly discriminative critic and easy-to-optimize objective, overcoming the mode-collapsing and brittle-training problems in existing methods. Extensive experiments are conducted on a variety of tasks to evaluate the proposed model empirically, including unconditional text generation, style transfer from non-parallel text, and unsupervised cipher cracking. The proposed model yields superior performance, demonstrating wide applicability and effectiveness.

Cited by in corpus (29)

• Evaluation of Text Generation: A Survey
• A survey on text generation using generative adversarial networks
• Defense Against Adversarial Attacks Using Feature Scattering-based Adversarial Training
• Gromov-Wasserstein Learning for Graph Matching and Node Embedding
• Topic-Guided Variational Autoencoders for Text Generation
• Neural Language Generation: Formulation, Methods, and Evaluation
• Learning by Semantic Similarity Makes Abstractive Summarization Better
• Adversarial Learning of Deepfakes in Accounting
• Self-Adversarial Learning with Comparative Discrimination for Text Generation
• Creative GANs for generating poems, lyrics, and metaphors
• Towards Generating Long and Coherent Text with Multi-Level Latent Variable Models
• ER-AE: Differentially Private Text Generation for Authorship Anonymization
• Improving Textual Network Embedding with Global Attention via Optimal Transport
• Rethinking Text Attribute Transfer: A Lexical Analysis
• Weakly supervised cross-domain alignment with optimal transport
• CatGAN: Category-aware Generative Adversarial Networks with Hierarchical Evolutionary Learning for Category Text Generation
• APo-VAE: Text Generation in Hyperbolic Space
• Conditional Hybrid GAN for Sequence Generation
• Improving Text Generation with Student-Forcing Optimal Transport
• Meta-CoTGAN: A Meta Cooperative Training Paradigm for Improving Adversarial Text Generation
• Math Word Problem Generation with Mathematical Consistency and Problem Context Constraints
• Generative Models for Security: Attacks, Defenses, and Opportunities
• The Detection of Distributional Discrepancy for Text Generation
• Syntax Matters! Syntax-Controlled in Text Style Transfer
• Adversarial Machine Learning in Text Analysis and Generation
• Generating Symbolic Reasoning Problems with Transformer GANs
• Improving Adversarial Text Generation by Modeling the Distant Future
• How Sequence-to-Sequence Models Perceive Language Styles?
• Sensing Ambiguity in Henry James' "The Turn of the Screw"