Geometry of Deep Convolutional Networks

Abstract

We give a formal procedure for computing preimages of convolutional network outputs using the dual basis defined from the set of hyperplanes associated with the layers of the network. We point out the special symmetry associated with arrangements of hyperplanes of convolutional networks that take the form of regular multidimensional polyhedral cones. We discuss the efficiency of large number of layers of nested cones that result from incremental small size convolutions in order to give a good compromise between efficient contraction of data to low dimensions and shaping of preimage manifolds. We demonstrate how a specific network flattens a non linear input manifold to an affine output manifold and discuss its relevance to understanding classification properties of deep networks.

Work in progress (to be updated)

References in corpus (5)

• Very Deep Convolutional Networks for Large-Scale Image Recognition
• On the Number of Linear Regions of Deep Neural Networks
• Visualizing Deep Convolutional Neural Networks Using Natural Pre-Images
• Understanding Deep Neural Networks with Rectified Linear Units
• Efficient Representation of Low-Dimensional Manifolds using Deep Networks

Cited by in corpus (1)

• Hyperplane Arrangements of Trained ConvNets Are Biased