Stochastic gradient descent on Riemannian manifolds
arXiv:1111.5280 · doi:10.1109/TAC.2013.2254619
Abstract
Stochastic gradient descent is a simple approach to find the local minima of a cost function whose evaluations are corrupted by noise. In this paper, we develop a procedure extending stochastic gradient descent algorithms to the case where the function is defined on a Riemannian manifold. We prove that, as in the Euclidian case, the gradient descent algorithm converges to a critical point of the cost function. The algorithm has numerous potential applications, and is illustrated here by four examples. In particular a novel gossip algorithm on the set of covariance matrices is derived and tested numerically.
arXiv commentA slightly shorter version has been published in IEEE Transactions Automatic Control
References in corpus (1)
Cited by in corpus (35)
- An elementary introduction to information geometry
- Riemannian conjugate gradient methods: General framework and specific algorithms with convergence analyses
- Riemannian stochastic variance reduced gradient algorithm with retraction and vector transport
- HRCF: Enhancing Collaborative Filtering via Hyperbolic Geometric Regularization
- Product Knowledge Graph Embedding for E-commerce
- HICF: Hyperbolic Informative Collaborative Filtering
- Hyperbolic Representation Learning for Fast and Efficient Neural Question Answering
- Walking on the Edge: Fast, Low-Distortion Adversarial Examples
- Beyond Convexity -- Contraction and Global Convergence of Gradient Descent
- Optimizing quantum circuits with Riemannian gradient flow
- Adaptive regularization with cubics on manifolds
- Performance of Hyperbolic Geometry Models on Top-N Recommendation Tasks
- Unsupervised Deep Metric Learning via Orthogonality based Probabilistic Loss
- Computation for Latent Variable Model Estimation: A Unified Stochastic Proximal Framework
- Hyperbolic Geometric Graph Representation Learning for Hierarchy-imbalance Node Classification
- Dual-Geometric Space Embedding Model for Two-View Knowledge Graphs
- Hyperbolic Node Embedding for Signed Networks
- Learning Co-Sparse Analysis Operators with Separable Structures
- Fast, asymptotically efficient, recursive estimation in a Riemannian manifold
- Joint Local Relational Augmentation and Global Nash Equilibrium for Federated Learning with Non-IID Data
- A stochastic algorithm finding -means on the circle
- Geometric Methods for Sampling, Optimisation, Inference and Adaptive Agents
- Riemannian Smoothing Gradient Type Algorithms]{Riemannian Smoothing Gradient Type Algorithms for Nonsmooth Optimization Problem on Compact Riemannian Submanifold Embedded in Euclidean Space
- Understanding and Mitigating Hyperbolic Dimensional Collapse in Graph Contrastive Learning
- Network Consensus in the Wasserstein Metric Space of Probability Measures
- A universal framework for learning the elliptical mixture model
- Efficient Random Walks on Riemannian Manifolds
- Linear Convergence of the Subspace Constrained Mean Shift Algorithm: From Euclidean to Directional Data
- Spherical Coordinates from Persistent Cohomology
- Riemannian Stochastic Variance-Reduced Cubic Regularized Newton Method for Submanifold Optimization
- Stochastic Modified Flows for Riemannian Stochastic Gradient Descent
- Convergence of variational Monte Carlo simulation and scale-invariant pre-training
- Hyperbolic recurrent neural network as the first type of non-Euclidean neural quantum state ansatz
- Stochastic Augmented Lagrangian Method in Riemannian Shape Manifolds
- Trainable Quantum Neural Network for Multiclass Image Classification with the Power of Pre-trained Tree Tensor Networks