Improving Model Training by Periodic Sampling over Weight Distributions

Samarth Tripathi, Jiayi Liu, Unmesh Kurup +2

In this paper, we explore techniques centered around periodic sampling of model weights that provide convergence improvements on gradient update methods (vanilla \acs{SGD}, Momentu…