Showing 2024Show all
3 papers · 1 filter
cs.LG2024
Explicit and Implicit Graduated Optimization in Deep Neural Networks
Naoki Sato, Hideaki Iiduka
Graduated optimization is a global optimization technique that is used to minimize a multimodal nonconvex function by smoothing the objective function with noise and gradually refi…
cs.LG2024
Scaled Conjugate Gradient Method for Nonconvex Optimization in Deep Neural Networks
Naoki Sato, Koshiro Izumi, Hideaki Iiduka
A scaled conjugate gradient method that accelerates existing adaptive methods utilizing stochastic gradients is proposed for solving nonconvex optimization problems with deep neura…
cs.LG2024
Momentum Does Not Reduce Stochastic Noise in Stochastic Gradient Descent
Naoki Sato, Hideaki Iiduka
For nonconvex objective functions, including those found in training deep neural networks, stochastic gradient descent (SGD) with momentum is said to converge faster and have bette…