Same Loss, Different Gradients

Ningkang Peng, Xiaoqian Peng, Yifan He +4

Differentiable learning typically assumes that the scalar objective evaluated in the forward pass and the gradient supplied to the optimizer in the backward pass describe the same…