Hierarchical visuomotor control of humanoids

Abstract

We aim to build complex humanoid agents that integrate perception, motor control, and memory. In this work, we partly factor this problem into low-level motor control from proprioception and high-level coordination of the low-level skills informed by vision. We develop an architecture capable of surprisingly flexible, task-directed motor control of a relatively high-DoF humanoid body by combining pre-training of low-level motor controllers with a high-level, task-focused controller that switches among low-level sub-policies. The resulting system is able to control a physically-simulated humanoid body to solve tasks that require coupling visual perception from an unstabilized egocentric RGB camera during locomotion in the environment. For a supplementary video link, see https://youtu.be/7GISvfbykLE .

arXiv commentAccepted as a conference paper at ICLR 2019

References in corpus (10)

• Continuous control with deep reinforcement learning
• DeepMimic: Example-Guided Deep Reinforcement Learning of Physics-Based Character Skills
• Human-level performance in first-person multiplayer games with population-based deep reinforcement learning
• Emergence of Locomotion Behaviours in Rich Environments
• IMPALA: Scalable Distributed Deep-RL with Importance Weighted Actor-Learner Architectures
• DeepMind Control Suite
• Learning Continuous Control Policies by Stochastic Value Gradients
• Learning human behaviors from motion capture by adversarial imitation
• Unsupervised Predictive Memory in a Goal-Directed Agent
• Learning and Transfer of Modulated Locomotor Controllers

Cited by in corpus (21)

• Convolutional Neural Networks as a Model of the Visual System: Past, Present, and Future
• dm_control: Software and Tasks for Continuous Control
• Critic Regularized Regression
• A Survey on Deep Learning for Skeleton-Based Human Animation
• Residual Force Control for Agile Human Behavior Imitation and Extended Motion Synthesis
• Robust Recovery Controller for a Quadrupedal Robot using Deep Reinforcement Learning
• Hierarchical Reinforcement Learning By Discovering Intrinsic Options
• From Pixels to Legs: Hierarchical Learning of Quadruped Locomotion
• UniCon: Universal Neural Controller For Physics-based Character Motion
• Sub-policy Adaptation for Hierarchical Reinforcement Learning
• Options as responses: Grounding behavioural hierarchies in multi-agent RL
• Guided Curriculum Learning for Walking Over Complex Terrain
• Local Search for Policy Iteration in Continuous Control
• Learning to Sit: Synthesizing Human-Chair Interactions via Hierarchical Control
• Adversarial Skill Chaining for Long-Horizon Robot Manipulation via Terminal State Regularization
• Thalamocortical motor circuit insights for more robust hierarchical control of complex sequences
• SimPoE: Simulated Character Control for 3D Human Pose Estimation
• Deep Learned Path Planning via Randomized Reward-Linked-Goals and Potential Space Applications
• Plan Arithmetic: Compositional Plan Vectors for Multi-Task Control
• Deep Reinforcement Learning for High Level Character Control
• Leveraging Sequentiality in Reinforcement Learning from a Single Demonstration