Representation Tradeoffs for Hyperbolic Embeddings

Abstract

Hyperbolic embeddings offer excellent quality with few dimensions when embedding hierarchical data structures like synonym or type hierarchies. Given a tree, we give a combinatorial construction that embeds the tree in hyperbolic space with arbitrarily low distortion without using optimization. On WordNet, our combinatorial embedding obtains a mean-average-precision of 0.989 with only two dimensions, while Nickel et al.'s recent construction obtains 0.87 using 200 dimensions. We provide upper and lower bounds that allow us to characterize the precision-dimensionality tradeoff inherent in any hyperbolic embedding. To embed general metric spaces, we propose a hyperbolic generalization of multidimensional scaling (h-MDS). We show how to perform exact recovery of hyperbolic points from distances, provide a perturbation analysis, and give a recovery result that allows us to reduce dimensionality. The h-MDS approach offers consistently low distortion even with few dimensions across several datasets. Finally, we extract lessons from the algorithms and theory above to design a PyTorch-based implementation that can handle incomplete information and is scalable.

Cited by in corpus (17)

• Network Geometry
• HICF: Hyperbolic Informative Collaborative Filtering
• Skip-gram word embeddings in hyperbolic space
• Hyperbolic Node Embedding for Signed Networks
• Embedding Node Structural Role Identity into Hyperbolic Space
• Knowledge Graph Embeddings and Explainable AI
• On Procrustes Analysis in Hyperbolic Space
• Knowledge Association with Hyperbolic Knowledge Graph Embeddings
• Zoo Guide to Network Embedding
• Incorporating Domain Knowledge into Health Recommender Systems using Hyperbolic Embeddings
• A unified Fourier slice method to derive ridgelet transform for a variety of depth-2 neural networks
• Hyperbolic Deep Learning for Foundation Models: A Survey
• Learning Hyperbolic Representations of Topological Features
• ${\text{H}}^2\text{TNE}$: Temporal Heterogeneous Information Network Embedding in Hyperbolic Spaces
• Augmenting the User-Item Graph with Textual Similarity Models
• Weisfeiler and Leman go Hyperbolic: Learning Distance Preserving Node Representations
• Siamese networks for Poincaré embeddings and the reconstruction of evolutionary trees