Beyond Backpropagation: Optimization with Multi-Tangent Forward Gradients

Katharina Flügel, Daniel Coquelin, Marie Weiel +3

The gradients used to train neural networks are typically computed using backpropagation. While an efficient way to obtain exact gradients, backpropagation is computationally expen…