The Free Energy Principle for Perception and Action: A Deep Learning Perspective

Abstract

The free energy principle, and its corollary active inference, constitute a bio-inspired theory that assumes biological agents act to remain in a restricted set of preferred states of the world, i.e., they minimize their free energy. Under this principle, biological agents learn a generative model of the world and plan actions in the future that will maintain the agent in an homeostatic state that satisfies its preferences. This framework lends itself to being realized in silico, as it comprehends important aspects that make it computationally affordable, such as variational inference and amortized planning. In this work, we investigate the tool of deep learning to design and realize artificial agents based on active inference, presenting a deep-learning oriented presentation of the free energy principle, surveying works that are relevant in both machine learning and active inference areas, and discussing the design choices that are involved in the implementation process. This manuscript probes newer perspectives for the active inference framework, grounding its theoretical aspects into more pragmatic affairs, offering a practical guide to active inference newcomers and a starting point for deep learning practitioners that would like to investigate implementations of the free energy principle.

References in corpus (53)

• An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale
• Auto-Encoding Variational Bayes
• Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modeling
• Convolutional LSTM Network: A Machine Learning Approach for Precipitation Nowcasting
• Representation Learning with Contrastive Predictive Coding
• Variational Inference: A Review for Statisticians
• Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actor
• Bootstrap your own latent: A new approach to self-supervised Learning
• Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learning
• Stochastic Backpropagation and Approximate Inference in Deep Generative Models
• The information bottleneck method
• Unsupervised Learning of Visual Features by Contrasting Cluster Assignments
• Estimating or Propagating Gradients Through Stochastic Neurons for Conditional Computation
• Variational Inference with Normalizing Flows
• Weight Uncertainty in Neural Networks
• Mastering Atari, Go, Chess and Shogi by Planning with a Learned Model
• Mastering Chess and Shogi by Self-Play with a General Reinforcement Learning Algorithm
• Alias-Free Generative Adversarial Networks
• Barlow Twins: Self-Supervised Learning via Redundancy Reduction
• The CMA Evolution Strategy: A Tutorial
• Solving Rubik's Cube with a Robot Hand
• VIME: Variational Information Maximizing Exploration
• Recurrent World Models Facilitate Policy Evolution
• Model-Based Reinforcement Learning for Atari
• NVAE: A Deep Hierarchical Variational Autoencoder
• Markov Chain Monte Carlo and Variational Inference: Bridging the Gap
• Large-Scale Study of Curiosity-Driven Learning
• Reinforcement Learning and Control as Probabilistic Inference: Tutorial and Review
• Learning Latent Dynamics for Planning from Pixels
• Stochastic Video Generation with a Learned Prior
• Concept Whitening for Interpretable Image Recognition
• Finding Approximate POMDP solutions Through Belief Compression
• Active inference on discrete state-spaces: a synthesis
• Memory-based control with recurrent neural networks
• Active inference: demystified and compared
• Dream to Control: Learning Behaviors by Latent Imagination
• Can stable and accurate neural networks be computed? -- On the barriers of deep learning and Smale's 18th problem
• From Variational to Deterministic Autoencoders
• Deep Active Inference
• Efficient Exploration via State Marginal Matching
• Active inference, eye movements and oculomotor delays
• Planning to Explore via Self-Supervised World Models
• Reinforcement Learning through Active Inference
• End-to-End Pixel-Based Deep Active Inference for Body Perception and Action
• Active Inference in Robotics and Artificial Agents: Survey and Challenges
• Some interesting observations on the free energy principle
• Unsupervised Control Through Non-Parametric Discriminative Rewards
• Model-Augmented Actor-Critic: Backpropagating through Paths
• Deep active inference agents using Monte-Carlo methods
• Bayesian policy selection using active inference
• Clockwork Variational Autoencoders
• Time Limits in Reinforcement Learning
• Exploration and preference satisfaction trade-off in reward-free learning

Cited by in corpus (5)

• Designing Ecosystems of Intelligence from First Principles
• Modeling motor control in continuous-time Active Inference: a survey
• On efficient computation in active inference
• Symmetry and Complexity in Object-Centric Deep Active Inference Models
• Recursive Neural Programs: Variational Learning of Image Grammars and Part-Whole Hierarchies