The Outer Product Structure of Neural Network Derivatives

Craig Bakker, Michael J. Henry, Nathan O. Hodas

In this paper, we show that feedforward and recurrent neural networks exhibit an outer product derivative structure but that convolutional neural networks do not. This structure ma…