Opening the black box of deep learning
arXiv:1805.08355
Abstract
The great success of deep learning shows that its technology contains profound truth, and understanding its internal mechanism not only has important implications for the development of its technology and effective application in various fields, but also provides meaningful insights into the understanding of human brain mechanism. At present, most of the theoretical research on deep learning is based on mathematics. This dissertation proposes that the neural network of deep learning is a physical system, examines deep learning from three different perspectives: microscopic, macroscopic, and physical world views, answers multiple theoretical puzzles in deep learning by using physics principles. For example, from the perspective of quantum mechanics and statistical physics, this dissertation presents the calculation methods for convolution calculation, pooling, normalization, and Restricted Boltzmann Machine, as well as the selection of cost functions, explains why deep learning must be deep, what characteristics are learned in deep learning, why Convolutional Neural Networks do not have to be trained layer by layer, and the limitations of deep learning, etc., and proposes the theoretical direction and basis for the further development of deep learning now and in the future. The brilliance of physics flashes in deep learning, we try to establish the deep learning technology based on the scientific theory of physics.
References in corpus (16)
- Recent Trends in Deep Learning Based Natural Language Processing
- Many-body quantum state tomography with neural networks
- Deep Learning: A Critical Appraisal
- Equivalence of restricted Boltzmann machines and tensor network states
- Quantum Entanglement in Deep Learning Architectures
- Quantum common causes and quantum causal models
- Mutual Information, Neural Networks and the Renormalization Group
- Large scale distributed neural network training through online distillation
- An exact mapping between the Variational Renormalization Group and Deep Learning
- Neural Network Renormalization Group
- What Do We Understand About Convolutional Networks?
- Mathematics of Deep Learning
- Deep Learning and Quantum Entanglement: Fundamental Connections with Implications to Network Design
- Autonomous Quantum Perceptron Neural Network
- Pooling is neither necessary nor sufficient for appropriate deformation stability in CNNs
- A Mathematical Framework for Superintelligent Machines