242 citations
- The University of TokyoJP23 papers
- Nagoya UniversityJP8 papers
- Hokkaido UniversityJP6 papers
- Institute of Space and Astronautical ScienceJP5 papers
- National Institute of Advanced Industrial Science and TechnologyJP5 papers
- Rikkyo UniversityJP5 papers
- Tohoku UniversityJP5 papers
- Chiba Institute of TechnologyJP4 papers
- Instituto de Astrofísica de CanariasES4 papers
- Japan Aerospace Exploration AgencyJP4 papers
- Kobe UniversityJP4 papers
- Kōchi UniversityJP4 papers
Showing math.OCShow all
2 papers · 1 filter
math.OC2021
The Number of Steps Needed for Nonconvex Optimization of a Deep Learning Optimizer is a Rational Function of Batch Size
Hideaki Iiduka
Recently, convergence as well as convergence rate analyses of deep learning optimizers for nonconvex optimization have been widely studied. Meanwhile, numerical evaluations for the…
math.OC2020★ 5 cited
Conjugate-gradient-based Adam for stochastic optimization and its application to deep learning
Yu Kobayashi, Hideaki Iiduka
This paper proposes a conjugate-gradient-based Adam algorithm blending Adam with nonlinear conjugate gradient methods and shows its convergence analysis. Numerical experiments on t…