Poincaré Embeddings for Learning Hierarchical Representations

Abstract

Representation learning has become an invaluable approach for learning from symbolic data such as text and graphs. However, while complex symbolic datasets often exhibit a latent hierarchical structure, state-of-the-art methods typically learn embeddings in Euclidean vector spaces, which do not account for this property. For this purpose, we introduce a new approach for learning hierarchical representations of symbolic data by embedding them into hyperbolic space -- or more precisely into an n-dimensional Poincaré ball. Due to the underlying hyperbolic geometry, this allows us to learn parsimonious representations of symbolic data by simultaneously capturing hierarchy and similarity. We introduce an efficient algorithm to learn the embeddings based on Riemannian optimization and show experimentally that Poincaré embeddings outperform Euclidean embeddings significantly on data with latent hierarchies, both in terms of representation capacity and in terms of generalization ability.

References in corpus (2)

• Hierarchical structure and the prediction of missing links in networks
• Sustaining the Internet with Hyperbolic Mapping

Cited by in corpus (12)

• Geoopt: Riemannian Optimization in PyTorch
• Performance of Hyperbolic Geometry Models on Top-N Recommendation Tasks
• A Perspective on Objects and Systematic Generalization in Model-Based RL
• Finer Grained Entity Typing with TypeNet
• The Tensor Brain: Semantic Decoding for Perception and Memory
• Embedding Cardinality Constraints in Neural Link Predictors
• ArGoT: A Glossary of Terms extracted from the arXiv
• Switch Spaces: Learning Product Spaces with Sparse Gating
• Target-Oriented Deformation of Visual-Semantic Embedding Space
• Representation Learning for Words and Entities
• Unsupervised Discovery of the Long-Tail in Instance Segmentation Using Hierarchical Self-Supervision
• Concept Formation and Dynamics of Repeated Inference in Deep Generative Models