247 citations · 967 across the 25 of their papers we have counts for
4 papers · 1 filter
Interpolating between Optimal Transport and MMD using Sinkhorn Divergences
Jean Feydy, Thibault Séjourné, François-Xavier Vialard +3
Comparing probability distributions is a fundamental problem in data sciences. Simple norms and divergences such as the total variation and the relative entropy only compare densit…
Fisher Information and Natural Gradient Learning of Random Deep Networks
Shun-ichi Amari, Ryo Karakida, Masafumi Oizumi
A deep neural network is a hierarchical nonlinear model transforming input signals to output signals. Its input-output relation is considered to be stochastic, being described for…
Statistical Neurodynamics of Deep Networks: Geometry of Signal Spaces
Shun-ichi Amari, Ryo Karakida, Masafumi Oizumi
Statistical neurodynamics studies macroscopic behaviors of randomly connected neural networks. We consider a deep layered feedforward network where input signals are processed laye…
Universal Statistics of Fisher Information in Deep Neural Networks: Mean Field Approach
Ryo Karakida, Shotaro Akaho, Shun-ichi Amari
The Fisher information matrix (FIM) is a fundamental quantity to represent the characteristics of a stochastic model, including deep neural networks (DNNs). The present study revea…