Theory of Generative Deep Learning : Probe Landscape of Empirical Error via Norm Based Capacity Control

Abstract

Despite its remarkable empirical success as a highly competitive branch of artificial intelligence, deep learning is often blamed for its widely known low interpretation and lack of firm and rigorous mathematical foundation. However, most theoretical endeavor is devoted in discriminative deep learning case, whose complementary part is generative deep learning. To the best of our knowledge, we firstly highlight landscape of empirical error in generative case to complete the full picture through exquisite design of image super resolution under norm based capacity control. Our theoretical advance in interpretation of the training dynamic is achieved from both mathematical and biological sides.

arXiv comment2018 5th IEEE International Conference on Cloud Computing and Intelligence Systems

References in corpus (7)

• Image Super-Resolution Using Deep Convolutional Networks
• Fisher-Rao Metric, Geometry, and Complexity of Neural Networks
• Representation Learning: A Review and New Perspectives
• How do neurons operate on sparse distributed representations? A mathematical theory of sparsity, neurons and active dendrites
• Deeply-Recursive Convolutional Network for Image Super-Resolution
• Theory II: Landscape of the Empirical Risk in Deep Learning
• Towards WARSHIP: Combining Components of Brain-Inspired Computing of RSH for Image Super Resolution