Poincaré Embeddings for Learning Hierarchical Representations

Abstract

Representation learning has become an invaluable approach for learning from symbolic data such as text and graphs. However, while complex symbolic datasets often exhibit a latent hierarchical structure, state-of-the-art methods typically learn embeddings in Euclidean vector spaces, which do not account for this property. For this purpose, we introduce a new approach for learning hierarchical representations of symbolic data by embedding them into hyperbolic space -- or more precisely into an n-dimensional Poincaré ball. Due to the underlying hyperbolic geometry, this allows us to learn parsimonious representations of symbolic data by simultaneously capturing hierarchy and similarity. We introduce an efficient algorithm to learn the embeddings based on Riemannian optimization and show experimentally that Poincaré embeddings outperform Euclidean embeddings significantly on data with latent hierarchies, both in terms of representation capacity and in terms of generalization ability.

References in corpus (6)

• Hierarchical structure and the prediction of missing links in networks
• HOGWILD!: A Lock-Free Approach to Parallelizing Stochastic Gradient Descent
• Complex Embeddings for Simple Link Prediction
• Sustaining the Internet with Hyperbolic Mapping
• Enriching Word Vectors with Subword Information
• Order-Embeddings of Images and Language

Cited by in corpus (27)

• A Review of Challenges and Opportunities in Machine Learning for Health
• Easing Embedding Learning by Comprehensive Transcription of Heterogeneous Information Networks
• Discriminative Topic Mining via Category-Name Guided Text Embedding
• Geoopt: Riemannian Optimization in PyTorch
• Beyond Convexity -- Contraction and Global Convergence of Gradient Descent
• Riemannian Continuous Normalizing Flows
• Performance of Hyperbolic Geometry Models on Top-N Recommendation Tasks
• Propositionalization and Embeddings: Two Sides of the Same Coin
• Embedding-based Silhouette Community Detection
• HyperKG: Hyperbolic Knowledge Graph Embeddings for Knowledge Base Completion
• A Perspective on Objects and Systematic Generalization in Model-Based RL
• Finer Grained Entity Typing with TypeNet
• Global and local evaluation of link prediction tasks with neural embeddings
• The Tensor Brain: Semantic Decoding for Perception and Memory
• Hyperbolic Deep Learning for Foundation Models: A Survey
• Embedding Ontologies via Incorporating Extensional and Intensional Knowledge
• ArGoT: A Glossary of Terms extracted from the arXiv
• Switch Spaces: Learning Product Spaces with Sparse Gating
• Embedding Cardinality Constraints in Neural Link Predictors
• A Geometric Perspective for High-Dimensional Multiplex Graphs
• Geometric Learning of Hidden Markov Models via a Method of Moments Algorithm
• Target-Oriented Deformation of Visual-Semantic Embedding Space
• Representation Learning for Words and Entities
• Siamese networks for Poincaré embeddings and the reconstruction of evolutionary trees
• Unsupervised Discovery of the Long-Tail in Instance Segmentation Using Hierarchical Self-Supervision
• CO-SNE: Dimensionality Reduction and Visualization for Hyperbolic Data
• Concept Formation and Dynamics of Repeated Inference in Deep Generative Models