Learning Robust Controllers Via Probabilistic Model-Based Policy Search

Abstract

Model-based Reinforcement Learning estimates the true environment through a world model in order to approximate the optimal policy. This family of algorithms usually benefits from better sample efficiency than their model-free counterparts. We investigate whether controllers learned in such a way are robust and able to generalize under small perturbations of the environment. Our work is inspired by the PILCO algorithm, a method for probabilistic policy search. We show that enforcing a lower bound to the likelihood noise in the Gaussian Process dynamics model regularizes the policy updates and yields more robust controllers. We demonstrate the empirical benefits of our method in a simulation benchmark.

arXiv commentAccepted at RobustML Workshop - ICLR 2021

References in corpus (7)

• Playing Atari with Deep Reinforcement Learning
• Offline Reinforcement Learning: Tutorial, Review, and Perspectives on Open Problems
• Robust Adversarial Reinforcement Learning
• Challenges of Real-World Reinforcement Learning
• Generalization in Reinforcement Learning with Selective Noise Injection and Information Bottleneck
• SAMBA: Safe Model-Based & Active Reinforcement Learning
• Learning Robust Options