PredNet and Predictive Coding: A Critical Review

Abstract

PredNet, a deep predictive coding network developed by Lotter et al., combines a biologically inspired architecture based on the propagation of prediction error with self-supervised representation learning in video. While the architecture has drawn a lot of attention and various extensions of the model exist, there is a lack of a critical analysis. We fill in the gap by evaluating PredNet both as an implementation of the predictive coding theory and as a self-supervised video prediction model using a challenging video action classification dataset. We design an extended model to test if conditioning future frame predictions on the action class of the video improves the model performance. We show that PredNet does not yet completely follow the principles of predictive coding. The proposed top-down conditioning leads to a performance gain on synthetic data, but does not scale up to the more complex real-world action classification dataset. Our analysis is aimed at guiding future research on similar architectures based on the predictive coding theory.

References in corpus (19)

• Convolutional LSTM Network: A Machine Learning Approach for Precipitation Nowcasting
• Representation Learning with Contrastive Predictive Coding
• Two-Stream Convolutional Networks for Action Recognition in Videos
• Unsupervised Learning of Video Representations using LSTMs
• Going Deeper with Convolutions
• Rethinking the Inception Architecture for Computer Vision
• Convolutional Two-Stream Network Fusion for Video Action Recognition
• PredRNN++: Towards A Resolution of the Deep-in-Time Dilemma in Spatiotemporal Predictive Learning
• Temporal Segment Networks: Towards Good Practices for Deep Action Recognition
• Fast R-CNN
• Self-supervised Visual Feature Learning with Deep Neural Networks: A Survey
• Learning to Generate Long-term Future via Hierarchical Prediction
• Deep Predictive Coding Network for Object Recognition
• On the effectiveness of task granularity for transfer learning
• Deep Predictive Coding Networks
• Inception-inspired LSTM for Next-frame Video Prediction
• Deep Predictive Coding Network with Local Recurrent Processing for Object Recognition
• AFA-PredNet: The action modulation within predictive coding
• Encoding Longer-term Contextual Multi-modal Information in a Predictive Coding Model

Cited by in corpus (4)

• Predictive Coding: a Theoretical and Experimental Review
• Inconsistent illusory motion in predictive coding deep neural networks
• Predictive coding, precision and natural gradients
• Variational Predictive Routing with Nested Subjective Timescales