CortexNet: a Generic Network Family for Robust Visual Temporal Representations
arXiv:1706.02735
Abstract
In the past five years we have observed the rise of incredibly well performing feed-forward neural networks trained supervisedly for vision related tasks. These models have achieved super-human performance on object recognition, localisation, and detection in still images. However, there is a need to identify the best strategy to employ these networks with temporal visual inputs and obtain a robust and stable representation of video data. Inspired by the human visual system, we propose a deep neural network family, CortexNet, which features not only bottom-up feed-forward connections, but also it models the abundant top-down feedback and lateral connections, which are present in our visual cortex. We introduce two training schemes - the unsupervised MatchNet and weakly supervised TempoNet modes - where a network learns how to correctly anticipate a subsequent frame in a video clip or the identity of its predominant subject, by learning egomotion clues and how to automatically track several objects in the current scene. Find the project website at https://engineering.purdue.edu/elab/CortexNet/.
8 pages, 4 figures. Edit: 4.2 - define n = t - 1; fix grammar/meaning in last sentence. 5.2 - add Open Images data set ref
References in corpus (7)
- An Analysis of Deep Neural Network Models for Practical Applications
- Deep Predictive Coding Networks for Video Prediction and Unsupervised Learning
- Explaining How a Deep Neural Network Trained with End-to-End Learning Steers a Car
- Spatio-temporal video autoencoder with differentiable memory
- Unsupervised Learning of Visual Representations using Videos
- Predicting Deeper into the Future of Semantic Segmentation
- Deep Predictive Coding Networks
Cited by in corpus (5)
- Deep Predictive Coding Network for Object Recognition
- Maximum A Posteriori Estimation of Distances Between Deep Features in Still-to-Video Face Recognition
- ToyArchitecture: Unsupervised Learning of Interpretable Models of the World
- Integrating Motion into Vision Models for Better Visual Prediction
- Variational Predictive Routing with Nested Subjective Timescales