Eval all, trust a few, do wrong to none: Comparing sentence generation models

Abstract

In this paper, we study recent neural generative models for text generation related to variational autoencoders. Previous works have employed various techniques to control the prior distribution of the latent codes in these models, which is important for sampling performance, but little attention has been paid to reconstruction error. In our study, we follow a rigorous evaluation protocol using a large set of previously used and novel automatic and human evaluation metrics, applied to both generated samples and reconstructions. We hope that it will become the new evaluation standard when comparing neural generative models for text.

arXiv comment12 pages (3 page appendix); v2: added hyperparameter settings, clarifications

References in corpus (10)

• Adam: A Method for Stochastic Optimization
• GANs Trained by a Two Time-Scale Update Rule Converge to a Local Nash Equilibrium
• Building Natural-Language Generation Systems
• Adversarial Autoencoders
• Learning Discourse-level Diversity for Neural Dialog Models using Conditional Variational Autoencoders
• Toward Controlled Generation of Text
• From optimal transport to generative modeling: the VEGAN cookbook
• Improved Variational Autoencoders for Text Modeling using Dilated Convolutions
• A Hybrid Convolutional Variational Autoencoder for Text Generation
• Generating Sentences by Editing Prototypes

Cited by in corpus (19)

• The Curious Case of Neural Text Degeneration
• Language GANs Falling Short
• On Accurate Evaluation of GANs for Language Generation
• Adversarially Regularized Autoencoders
• A Tutorial on Deep Latent Variable Models of Natural Language
• Educating Text Autoencoders: Latent Representation Guidance via Denoising
• How Faithful is your Synthetic Data? Sample-level Metrics for Evaluating and Auditing Generative Models
• SALSA-TEXT : self attentive latent space based adversarial text generation
• Towards Generating Long and Coherent Text with Multi-Level Latent Variable Models
• Deleter: Leveraging BERT to Perform Unsupervised Successive Text Compression
• Judge the Judges: A Large-Scale Evaluation Study of Neural Language Models for Online Review Generation
• Learning Latent Space Energy-Based Prior Model
• Plug and Play Autoencoders for Conditional Text Generation
• Sparse Text Generation
• Multi-scale Transformer Language Models
• Adding A Filter Based on The Discriminator to Improve Unconditional Text Generation
• Language Model Evaluation in Open-ended Text Generation
• Adversarial Machine Learning in Text Analysis and Generation
• Fiction Sentence Expansion and Enhancement via Focused Objective and Novelty Curve Sampling