A backward pass through a CNN using a generative model of its activations

Abstract

Neural networks have shown to be a practical way of building a very complex mapping between a pre-specified input space and output space. For example, a convolutional neural network (CNN) mapping an image into one of a thousand object labels is approaching human performance in this particular task. However the mapping (neural network) does not automatically lend itself to other forms of queries, for example, to detect/reconstruct object instances, to enforce top-down signal on ambiguous inputs, or to recover object instances from occlusion. One way to address these queries is a backward pass through the network that fuses top-down and bottom-up information. In this paper, we show a way of building such a backward pass by defining a generative model of the neural network's activations. Approximate inference of the model would naturally take the form of a backward pass through the CNN layers, and it addresses the aforementioned queries in a unified framework.

References in corpus (7)

• Very Deep Convolutional Networks for Large-Scale Image Recognition
• Striving for Simplicity: The All Convolutional Net
• DRAW: A Recurrent Neural Network For Image Generation
• Multiple Object Recognition with Visual Attention
• Attend, Infer, Repeat: Fast Scene Understanding with Generative Models
• Stacked What-Where Auto-encoders
• Towards Scene Understanding with Detailed 3D Object Representations