Learning Robust and Adaptive Real-World Continuous Control Using Simulation and Transfer Learning

Abstract

We use model-free reinforcement learning, extensive simulation, and transfer learning to develop a continuous control algorithm that has good zero-shot performance in a real physical environment. We train a simulated agent to act optimally across a set of similar environments, each with dynamics drawn from a prior distribution. We propose that the agent is able to adjust its actions almost immediately, based on small set of observations. This robust and adaptive behavior is enabled by using a policy gradient algorithm with an Long Short Term Memory (LSTM) function approximation. Finally, we train an agent to navigate a two-dimensional environment with uncertain dynamics and noisy observations. We demonstrate that this agent has good zero-shot performance in a real physical environment. Our preliminary results indicate that the agent is able to infer the environmental dynamics after only a few timesteps, and adjust its actions accordingly.

arXiv commentThe paper has several technical errors. Rather than correct these errors we have chosen to significantly reformulate the work

References in corpus (7)

• Continuous control with deep reinforcement learning
• Overcoming catastrophic forgetting in neural networks
• Benchmarking Deep Reinforcement Learning for Continuous Control
• Residual Networks Behave Like Ensembles of Relatively Shallow Networks
• Scalable trust-region method for deep reinforcement learning using Kronecker-factored approximation
• Sample Efficient Actor-Critic with Experience Replay
• Transfer from Simulation to Real World through Learning Deep Inverse Dynamics Model