How deep learning works --The geometry of deep learning
arXiv:1710.10784
Abstract
Why and how that deep learning works well on different tasks remains a mystery from a theoretical perspective. In this paper we draw a geometric picture of the deep learning system by finding its analogies with two existing geometric structures, the geometry of quantum computations and the geometry of the diffeomorphic template matching. In this framework, we give the geometric structures of different deep learning systems including convolutional neural networks, residual networks, recursive neural networks, recurrent neural networks and the equilibrium prapagation framework. We can also analysis the relationship between the geometrical structures and their performance of different networks in an algorithmic level so that the geometric framework may guide the design of the structures and algorithms of deep learning systems.
arXiv comment16 pages, 13 figures
References in corpus (15)
- A Critical Review of Recurrent Neural Networks for Sequence Learning
- Building up spacetime with quantum entanglement
- Complexity and Shock Wave Geometries
- Quantum Computation as Geometry
- Grid Long Short-Term Memory
- Exponential expressivity in deep neural networks through transient chaos
- Identity Mappings in Deep Residual Networks
- Switchbacks and the Bridge to Nowhere
- Reinforcement Learning Neural Turing Machines - Revised
- The geometry of entanglement: metrics, connections and the geometric phase
- The momentum map representation of images
- The twistor geometry of three-qubit entanglement
- Geometrical Formulation of Quantum Mechanics
- Geometric formulation of quantum mechanics
- Equilibrium Propagation: Bridging the Gap Between Energy-Based Models and Backpropagation
Cited by in corpus (6)
- Geometrization of deep networks for the interpretability of deep learning systems
- Spacetime as the optimal generative network of quantum states: a roadmap to QM=GR?
- Demystifying AlphaGo Zero as AlphaGo GAN
- Understanding over-parameterized deep networks by geometrization
- Deep network as memory space: complexity, generalization, disentangled representation and interpretability
- Gauge theory and twins paradox of disentangled representations