Improving the Convergence Rates of Forward Gradient Descent with Repeated Sampling

Niklas Dexheimer, Johannes Schmidt-Hieber

Forward gradient descent (FGD) has been proposed as a biologically more plausible alternative of gradient descent as it can be computed without backward pass. Considering the linea…