Second-order Neural Network Training Using Complex-step Directional Derivative

Siyuan Shen, Tianjia Shao, Kun Zhou +3

While the superior performance of second-order optimization methods such as Newton's method is well known, they are hardly used in practice for deep learning because neither assemb…