Deep Function Machines: Generalized Neural Networks for Topological Layer Expression
arXiv:1612.04799
Abstract
In this paper we propose a generalization of deep neural networks called deep function machines (DFMs). DFMs act on vector spaces of arbitrary (possibly infinite) dimension and we show that a family of DFMs are invariant to the dimension of input data; that is, the parameterization of the model does not directly hinge on the quality of the input (eg. high resolution images). Using this generalization we provide a new theory of universal approximation of bounded non-linear operators between function spaces. We then suggest that DFMs provide an expressive framework for designing new neural network layer types with topological considerations in mind. Finally, we introduce a novel architecture, RippLeNet, for resolution invariant computer vision, which empirically achieves state of the art invariance.
arXiv comment23 pages, 9 figures, with experiments
References in corpus (7)
- Deep Learning in Neural Networks: An Overview
- Toward Deeper Understanding of Neural Networks: The Power of Initialization and a Dual View on Expressivity
- On the Expressive Power of Deep Neural Networks
- Analyzing the Performance of Multilayer Neural Networks for Object Recognition
- Steps Toward Deep Kernel Methods from Infinite Neural Networks
- Analysis and Extension of Arc-Cosine Kernels for Large Margin Classification
- Learning Infinite-Layer Networks: Without the Kernel Trick