Understanding over-parameterized deep networks by geometrization
arXiv:1902.03793
Abstract
A complete understanding of the widely used over-parameterized deep networks is a key step for AI. In this work we try to give a geometric picture of over-parameterized deep networks using our geometrization scheme. We show that the Riemannian geometry of network complexity plays a key role in understanding the basic properties of over-parameterizaed deep networks, including the generalization, convergence and parameter sensitivity. We also point out deep networks share lots of similarities with quantum computation systems. This can be regarded as a strong support of our proposal that geometrization is not only the bible for physics, it is also the key idea to understand deep learning systems.
6 pages
References in corpus (16)
- A Convergence Theory for Deep Learning via Over-Parameterization
- Quantum Computation as Geometry
- Gradient Descent Provably Optimizes Over-parameterized Neural Networks
- No bad local minima: Data independent training error guarantees for multilayer neural networks
- On the Optimization of Deep Networks: Implicit Acceleration by Overparameterization
- Towards Understanding Generalization of Deep Learning: Perspective of Loss Landscapes
- Switchbacks and the Bridge to Nowhere
- Fisher-Rao Metric, Geometry, and Complexity of Neural Networks
- The Typical-State Paradox: Diagnosing Horizons with Complexity
- The momentum map representation of images
- The loss landscape of overparameterized neural networks
- Geometric formulation of quantum mechanics
- How deep learning works --The geometry of deep learning
- Emergent General Relativity from Fisher Information Metric
- Geometrization of deep networks for the interpretability of deep learning systems
- Spacetime as the optimal generative network of quantum states: a roadmap to QM=GR?