End-to-End Pixel-Based Deep Active Inference for Body Perception and Action

Abstract

We present a pixel-based deep active inference algorithm (PixelAI) inspired by human body perception and action. Our algorithm combines the free-energy principle from neuroscience, rooted in variational inference, with deep convolutional decoders to scale the algorithm to directly deal with raw visual input and provide online adaptive inference. Our approach is validated by studying body perception and action in a simulated and a real Nao robot. Results show that our approach allows the robot to perform 1) dynamical body estimation of its arm using only monocular camera images and 2) autonomous reaching to "imagined" arm poses in the visual space. This suggests that robot and human body perception and action can be efficiently solved by viewing both as an active inference problem guided by ongoing sensory input.

References in corpus (6)

• Auto-Encoding Variational Bayes
• End-to-End Training of Deep Visuomotor Policies
• A guide to convolution arithmetic for deep learning
• Visual Reinforcement Learning with Imagined Goals
• Drifting perceptual patterns suggest prediction errors fusion rather than hypothesis selection: replicating the rubber-hand illusion on a robot
• Sensorimotor learning for artificial body perception

Cited by in corpus (12)

• Active inference body perception and action for humanoid robots
• The Free Energy Principle for Perception and Action: A Deep Learning Perspective
• Neural dynamics under active inference: plausibility and efficiency of information processing
• Robot in the mirror: toward an embodied computational model of mirror self-recognition
• Deep Active Inference for Partially Observable MDPs
• Robot self/other distinction: active inference meets neural networks learning in a mirror
• Bayesian Mechanics for Stationary Processes
• Modeling motor control in continuous-time Active Inference: a survey
• A deep active inference model of the rubber-hand illusion
• Active Inference Tree Search in Large POMDPs
• Neuroscience-inspired perception-action in robotics: applying active inference for state estimation, control and self-perception
• Visual Sensation and Perception Computational Models for Deep Learning: State of the art, Challenges and Prospects