5 citations · 5 across the 3 of their papers we have counts for
7 papers
The Number of Steps Needed for Nonconvex Optimization of a Deep Learning Optimizer is a Rational Function of Batch Size
Hideaki Iiduka
Recently, convergence as well as convergence rate analyses of deep learning optimizers for nonconvex optimization have been widely studied. Meanwhile, numerical evaluations for the…
Riemannian Stochastic Fixed Point Optimization Algorithm
Hideaki Iiduka, Hiroyuki Sakai
This paper considers a stochastic optimization problem over the fixed point sets of quasinonexpansive mappings on Riemannian manifolds. The problem enables us to consider Riemannia…
Riemannian Adaptive Optimization Algorithm and Its Application to Natural Language Processing
Hiroyuki Sakai, Hideaki Iiduka
This paper proposes a Riemannian adaptive optimization algorithm to optimize the parameters of deep neural networks. The algorithm is an extension of both AMSGrad in Euclidean spac…
Conjugate-gradient-based Adam for stochastic optimization and its application to deep learning
Yu Kobayashi, Hideaki Iiduka
This paper proposes a conjugate-gradient-based Adam algorithm blending Adam with nonlinear conjugate gradient methods and shows its convergence analysis. Numerical experiments on t…
Appropriate Learning Rates of Adaptive Learning Rate Optimization Algorithms for Training Deep Neural Networks
Hideaki Iiduka
This paper deals with nonconvex stochastic optimization problems in deep learning and provides appropriate learning rates with which adaptive learning rate optimization algorithms,…
Hybrid Riemannian Conjugate Gradient Methods with Global Convergence Properties
Hiroyuki Sakai, Hideaki Iiduka
This paper presents new Riemannian conjugate gradient methods and global convergence analyses under the strong Wolfe conditions. The main idea of the new methods is to combine the…