Hyperbolic Neural Networks

Abstract

Hyperbolic spaces have recently gained momentum in the context of machine learning due to their high capacity and tree-likeliness properties. However, the representational power of hyperbolic geometry is not yet on par with Euclidean geometry, mostly because of the absence of corresponding hyperbolic neural network layers. This makes it hard to use hyperbolic embeddings in downstream tasks. Here, we bridge this gap in a principled manner by combining the formalism of Möbius gyrovector spaces with the Riemannian geometry of the Poincaré model of hyperbolic spaces. As a result, we derive hyperbolic versions of important deep learning tools: multinomial logistic regression, feed-forward and recurrent neural networks such as gated recurrent units. This allows to embed sequential data and perform classification in the hyperbolic space. Empirically, we show that, even if hyperbolic optimization tools are limited, hyperbolic sentence embeddings either outperform or are on par with their Euclidean variants on textual entailment and noisy-prefix recognition tasks.

References in corpus (3)

• A large annotated corpus for learning natural language inference
• Hyperbolic Entailment Cones for Learning Hierarchical Embeddings
• Can recurrent neural networks warp time?

Cited by in corpus (19)

• The Universal Approximation Property
• Geoopt: Riemannian Optimization in PyTorch
• Self-Supervised Hyperboloid Representations from Logical Queries over Knowledge Graphs
• Where are we in embedding spaces? A Comprehensive Analysis on Network Embedding Approaches for Recommender Systems
• Skip-gram word embeddings in hyperbolic space
• Poincaré Wasserstein Autoencoder
• Hyperbolic Interaction Model For Hierarchical Multi-Label Classification
• HCGR: Hyperbolic Contrastive Graph Representation Learning for Session-based Recommendation
• Manifolds for Unsupervised Visual Anomaly Detection
• Hyperbolic Deep Learning for Chinese Natural Language Understanding
• Switch Spaces: Learning Product Spaces with Sparse Gating
• Stealing Black-Box Functionality Using The Deep Neural Tree Architecture
• Augmenting the User-Item Graph with Textual Similarity Models
• Hyperbolic Hypergraphs for Sequential Recommendation
• Conformal retrofitting via Riemannian manifolds: distilling task-specific graphs into pretrained embeddings
• Hyperbolic Manifold Regression
• An Informal Introduction to Multiplet Neural Networks
• THG: Transformer with Hyperbolic Geometry
• CO-SNE: Dimensionality Reduction and Visualization for Hyperbolic Data