APo-VAE: Text Generation in Hyperbolic Space
arXiv:2005.00054
Abstract
Natural language often exhibits inherent hierarchical structure ingrained with complex syntax and semantics. However, most state-of-the-art deep generative models learn embeddings only in Euclidean vector space, without accounting for this structural property of language. In this paper, we investigate text generation in a hyperbolic latent space to learn continuous hierarchical representations. An Adversarial Poincare Variational Autoencoder (APo-VAE) is presented, where both the prior and variational posterior of latent variables are defined over a Poincare ball via wrapped normal distributions. By adopting the primal-dual formulation of KL divergence, an adversarial learning procedure is introduced to empower robust model training. Extensive experiments in language modeling and dialog-response generation tasks demonstrate the winning effectiveness of the proposed APo-VAE model over VAEs in Euclidean latent space, thanks to its superb capabilities in capturing latent language hierarchies in hyperbolic space.
References in corpus (16)
- Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translation
- Style Transfer from Non-Parallel Text by Cross-Alignment
- How NOT To Evaluate Your Dialogue System: An Empirical Study of Unsupervised Evaluation Metrics for Dialogue Response Generation
- InfoVAE: Information Maximizing Variational Autoencoders
- Adversarial Variational Bayes: Unifying Variational Autoencoders and Generative Adversarial Networks
- Poincaré GloVe: Hyperbolic Word Embeddings
- Lagging Inference Networks and Posterior Collapse in Variational Autoencoders
- Adversarial Text Generation via Feature-Mover's Distance
- Topic-Guided Variational Autoencoders for Text Generation
- Neural Embeddings of Graphs in Hyperbolic Space
- ControlVAE: Controllable Variational Autoencoder
- Continuous Hierarchical Representations with Poincaré Variational Auto-Encoders
- Latent Normalizing Flows for Discrete Sequences
- Adversarial Autoencoders with Constant-Curvature Latent Manifolds
- Poincaré Wasserstein Autoencoder
- Improving Textual Network Learning with Variational Homophilic Embeddings