247 citations · 967 across the 25 of their papers we have counts for
Showing 2020 · stat.MLShow all
2 papers · 2 filters
stat.ML2020★ 11 cited
When Does Preconditioning Help or Hurt Generalization?
Shun-ichi Amari, Jimmy Ba, Roger Grosse +5
While second order optimizers such as natural gradient descent (NGD) often speed up optimization, their effect on generalization has been called into question. This work presents a…
stat.ML2020★ 3 cited
Any Target Function Exists in a Neighborhood of Any Sufficiently Wide Random Network: A Geometrical Perspective
Shun-ichi Amari
It is known that any target function is realized in a sufficiently small neighborhood of any randomly connected deep network, provided the width (the number of neurons in a layer)…