Torque-Aware Momentum

Pranshu Malviya, Goncalo Mordido, Aristide Baratin +4

Efficiently exploring complex loss landscapes is key to the performance of deep neural networks. While momentum-based optimizers are widely used in state-of-the-art setups, classic…