Kalman Gradient Descent: Adaptive Variance Reduction in Stochastic Optimization

James Vuckovic

We introduce Kalman Gradient Descent, a stochastic optimization algorithm that uses Kalman filtering to adaptively reduce gradient variance in stochastic gradient descent by filter…