Stein-Rule Shrinkage for Stochastic Gradient Estimation in High Dimensions

M. Arashi, M. Amintoosi

Stochastic gradient methods are central to large-scale learning, but they treat mini-batch gradients as unbiased estimators, which classical decision theory shows are inadmissible…