Gradient descent in hyperbolic space

Abstract

Gradient descent generalises naturally to Riemannian manifolds, and to hyperbolic $n$-space, in particular. Namely, having calculated the gradient at the point on the manifold representing the model parameters, the updated point is obtained by travelling along the geodesic passing in the direction of the gradient. Some recent works employing optimisation in hyperbolic space have not attempted this procedure, however, employing instead various approximations to avoid a calculation that was considered to be too complicated. In this tutorial, we demonstrate that in the hyperboloid model of hyperbolic space, the necessary calculations to perform gradient descent are in fact straight-forward. The advantages of the approach are then both illustrated and quantified for the optimisation problem of computing the Fréchet mean (i.e. barycentre) of points in hyperbolic space.

arXiv comment10 pages

References in corpus (2)

• Neural Embeddings of Graphs in Hyperbolic Space
• Towards Riemannian Accelerated Gradient Methods

Cited by in corpus (11)

• Skip-gram word embeddings in hyperbolic space
• HyperML: A Boosting Metric Learning Approach in Hyperbolic Space for Recommender Systems
• Scalable Hyperbolic Recommender Systems
• Hyperbolic Interaction Model For Hierarchical Multi-Label Classification
• Unsupervised Hierarchy Matching with Optimal Transport over Hyperbolic Spaces
• Hyperbolic Deep Learning for Chinese Natural Language Understanding
• Embedding Heterogeneous Networks into Hyperbolic Space Without Meta-path
• Learning phylogenetic trees as hyperbolic point configurations
• Lorentzian Graph Convolutional Networks
• HEAT: Hyperbolic Embedding of Attributed Networks
• Overlapping Spaces for Compact Graph Representations