Learning Dynamics Reveal a Hierarchy of Weight-Induced Layerwise Gram Metrics

Claudio Nordio

We study feed-forward ReLU networks with fixed readout and quadratic loss, and rewrite gradient descent as a collective dynamics of activation fields and conjugate fields on the tr…