Emergence of Complex-Like Cells in a Temporal Product Network with Local Receptive Fields

Abstract

We introduce a new neural architecture and an unsupervised algorithm for learning invariant representations from temporal sequence of images. The system uses two groups of complex cells whose outputs are combined multiplicatively: one that represents the content of the image, constrained to be constant over several consecutive frames, and one that represents the precise location of features, which is allowed to vary over time but constrained to be sparse. The architecture uses an encoder to extract features, and a decoder to reconstruct the input from the features. The method was applied to patches extracted from consecutive movie frames and produces orientation and frequency selective units analogous to the complex cells in V1. An extension of the method is proposed to train a network composed of units with local receptive field spread over a large image of arbitrary size. A layer of complex cells, subject to sparsity constraints, pool feature units over overlapping local neighborhoods, which causes the feature units to organize themselves into pinwheel patterns of orientation-selective receptive fields, similar to those observed in the mammalian visual cortex. A feed-forward encoder efficiently computes the feature representation of full images.

References in corpus (1)

• Fast Inference in Sparse Coding Algorithms with Applications to Object Recognition

Cited by in corpus (20)

• Spectral Networks and Locally Connected Networks on Graphs
• Convolutional Neural Networks on Graphs with Fast Localized Spectral Filtering
• Deep Convolutional Networks on Graph-Structured Data
• Predicting Parameters in Deep Learning
• Video (language) modeling: a baseline for generative models of natural videos
• Building high-level features using large scale unsupervised learning
• Representation Learning: A Review and New Perspectives
• Face Recognition: From Traditional to Deep Learning Methods
• Deep AutoRegressive Networks
• Automated Pruning for Deep Neural Network Compression
• Texture Modeling with Convolutional Spike-and-Slab RBMs and Deep Extensions
• Revisiting Spatial Invariance with Low-Rank Local Connectivity
• Dynamic Region-Aware Convolution
• Efficient Learning of Sparse Invariant Representations
• Auto-pooling: Learning to Improve Invariance of Image Features from Image Sequences
• Learning Spatiotemporal Features of Ride-sourcing Services with Fusion Convolutional Network
• Locally Smoothed Neural Networks
• Semi-Local Convolutions for LiDAR Scan Processing
• Convolution in Convolution for Network in Network
• On the Exploration of Convolutional Fusion Networks for Visual Recognition