Synaptic Dynamics Realize First-order Adaptive Learning and Weight Symmetry

Yukun Yang, Peng Li

Gradient-based first-order adaptive optimization methods such as the Adam optimizer are prevalent in training artificial networks, achieving the state-of-the-art results. This work…