10 papers
Mini-Batch Stochastic Halpern Algorithm for Nonexpansive Fixed Point Problems
Hideaki Iiduka
The Halpern algorithm is a powerful fixed point approximation method for finding the closest point in the fixed point set of a nonexpansive mapping to the initial point. However, i…
Mini-Batch Stochastic Krasnosel'ski\uı-Mann Algorithm for Nonexpansive Fixed Point Problems
Hideaki Iiduka
The Krasnosel'ski\uı-Mann algorithm is a well-known method for finding fixed points of a nonexpansive mapping with strong theoretical guarantees. However, there are practical larg…
Faster Convergence of Riemannian Stochastic Gradient Descent with Increasing Batch Size
Kanata Oowada, Hideaki Iiduka
We theoretically analyzed the convergence behavior of Riemannian stochastic gradient descent (RSGD) and found that using an increasing batch size leads to faster convergence than u…
Accelerating SGDM via Learning Rate and Batch Size Schedules: A Lyapunov-Based Analysis
Yuichi Kondo, Hideaki Iiduka
We analyze the convergence behavior of stochastic gradient descent with momentum (SGDM) under dynamic learning-rate and batch-size schedules by introducing a novel and simpler Lyap…
Increasing Batch Size Improves Convergence of Stochastic Gradient Descent with Momentum
Keisuke Kamo, Hideaki Iiduka
Stochastic gradient descent with momentum (SGDM), in which a momentum term is added to SGD, has been well studied in both theory and practice. The theoretical studies show that the…
Both Asymptotic and Non-Asymptotic Convergence of Quasi-Hyperbolic Momentum using Increasing Batch Size
Kento Imaizumi, Hideaki Iiduka
Momentum methods were originally introduced for their superiority to stochastic gradient descent (SGD) in deterministic settings with convex objective functions. However, despite t…