45 citations · 93 across the 25 of their papers we have counts for
3 papers · 1 filter
Fisher Information and Natural Gradient Learning of Random Deep Networks
Shun-ichi Amari, Ryo Karakida, Masafumi Oizumi
A deep neural network is a hierarchical nonlinear model transforming input signals to output signals. Its input-output relation is considered to be stochastic, being described for…
Statistical Neurodynamics of Deep Networks: Geometry of Signal Spaces
Shun-ichi Amari, Ryo Karakida, Masafumi Oizumi
Statistical neurodynamics studies macroscopic behaviors of randomly connected neural networks. We consider a deep layered feedforward network where input signals are processed laye…
Universal Statistics of Fisher Information in Deep Neural Networks: Mean Field Approach
Ryo Karakida, Shotaro Akaho, Shun-ichi Amari
The Fisher information matrix (FIM) is a fundamental quantity to represent the characteristics of a stochastic model, including deep neural networks (DNNs). The present study revea…