Joint Perception and Control as Inference with an Object-based Implementation

Abstract

Existing model-based reinforcement learning methods often study perception modeling and decision making separately. We introduce joint Perception and Control as Inference (PCI), a general framework to combine perception and control for partially observable environments through Bayesian inference. Based on the fact that object-level inductive biases are critical in human perceptual learning and reasoning, we propose Object-based Perception Control (OPC), an instantiation of PCI which manages to facilitate control using automatic discovered object-based representations. We develop an unsupervised end-to-end solution and analyze the convergence of the perception model update. Experiments in a high-dimensional pixel environment demonstrate the learning effectiveness of our object-based perception control approach. Specifically, we show that OPC achieves good perceptual grouping quality and outperforms several strong baselines in accumulated rewards.

References in corpus (17)

• Adam: A Method for Stochastic Optimization
• Playing Atari with Deep Reinforcement Learning
• Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actor
• Recurrent World Models Facilitate Policy Evolution
• Reinforcement Learning and Control as Probabilistic Inference: Tutorial and Review
• Continuous Deep Q-Learning with Model-based Acceleration
• Optimal control as a graphical model inference problem
• Multi-Object Representation Learning with Iterative Variational Inference
• Relational Deep Reinforcement Learning
• Towards Deep Symbolic Reinforcement Learning
• Unsupervised Learning of Object Keypoints for Perception and Control
• Efficient Exploration via State Marginal Matching
• COBRA: Data-Efficient Model-Based RL through Unsupervised Object Discovery and Curiosity-Driven Exploration
• Unsupervised Video Object Segmentation for Deep Reinforcement Learning
• Action and Perception as Divergence Minimization
• Object-Oriented Dynamics Predictor
• Approximate Inference and Stochastic Optimal Control