Altering Backward Pass Gradients improves Convergence

Bishshoy Das, Milton Mondal, Brejesh Lall +2

In standard neural network training, the gradients in the backward pass are determined by the forward pass. As a result, the two stages are coupled. This is how most neural network…