Hyperbolic Deep Neural Networks: A Survey
arXiv:2101.04562
Abstract
Recently, there has been a rising surge of momentum for deep representation learning in hyperbolic spaces due to theirhigh capacity of modeling data like knowledge graphs or synonym hierarchies, possessing hierarchical structure. We refer to the model as hyperbolic deep neural network in this paper. Such a hyperbolic neural architecture potentially leads to drastically compact model withmuch more physical interpretability than its counterpart in Euclidean space. To stimulate future research, this paper presents acoherent and comprehensive review of the literature around the neural components in the construction of hyperbolic deep neuralnetworks, as well as the generalization of the leading deep approaches to the Hyperbolic space. It also presents current applicationsaround various machine learning tasks on several publicly available datasets, together with insightful observations and identifying openquestions and promising future directions.
References in corpus (43)
- Very Deep Convolutional Networks for Large-Scale Image Recognition
- An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale
- The structure and function of complex networks
- PyTorch: An Imperative Style, High-Performance Deep Learning Library
- Auto-Encoding Variational Bayes
- Semi-Supervised Classification with Graph Convolutional Networks
- Power laws, Pareto distributions and Zipf's law
- ADADELTA: An Adaptive Learning Rate Method
- Prototypical Networks for Few-shot Learning
- Playing Atari with Deep Reinforcement Learning
- Geometric deep learning: going beyond Euclidean data
- Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translation
- Instance Normalization: The Missing Ingredient for Fast Stylization
- Spectral Networks and Locally Connected Networks on Graphs
- On the Convergence of Adam and Beyond
- Deep Convolutional Networks on Graph-Structured Data
- Variational Inference with Normalizing Flows
- Normalizing Flows: An Introduction and Review of Current Methods
- Hyperbolic Geometry of Complex Networks
- Person Re-identification: Past, Present and Future
- Density estimation using Real NVP
- Pitfalls of Graph Neural Network Evaluation
- Manifold Mixup: Better Representations by Interpolating Hidden States
- Riemannian Adaptive Optimization Methods
- On Differentiating Parameterized Argmin and Argmax Problems with Application to Bi-level Optimization
- Hyperbolic Graph Neural Networks
- Neural Embeddings of Graphs in Hyperbolic Space
- Higher-Order Factorization Machines
- Geoopt: Riemannian Optimization in PyTorch
- Hyperbolic Attention Networks
- Normalizing Flows on Riemannian Manifolds
- Adversarial Autoencoders with Constant-Curvature Latent Manifolds
- Multi-relational Poincaré Graph Embeddings
- Poincaré Wasserstein Autoencoder
- Scalable Hyperbolic Recommender Systems
- HyperKG: Hyperbolic Knowledge Graph Embeddings for Knowledge Base Completion
- Hyperbolic Graph Attention Network
- Latent Variable Modelling with Hyperbolic Normalizing Flows
- APo-VAE: Text Generation in Hyperbolic Space
- Mix Dimension in Poincaré Geometry for 3D Skeleton-based Action Recognition
- Hyperbolic Deep Learning for Chinese Natural Language Understanding
- Are Hyperbolic Representations in Graphs Created Equal?
- Tiling the Euclidean and Hyperbolic planes with ribbons