Evolving Rewards to Automate Reinforcement Learning
arXiv:1905.07628
Abstract
Many continuous control tasks have easily formulated objectives, yet using them directly as a reward in reinforcement learning (RL) leads to suboptimal policies. Therefore, many classical control tasks guide RL training using complex rewards, which require tedious hand-tuning. We automate the reward search with AutoRL, an evolutionary layer over standard RL that treats reward tuning as hyperparameter optimization and trains a population of RL agents to find a reward that maximizes the task objective. AutoRL, evaluated on four Mujoco continuous control tasks over two RL algorithms, shows improvements over baselines, with the the biggest uplift for more complex tasks. The video can be found at: \url{https://youtu.be/svdaOFfQyC8}.
Accepted to 6th AutoML@ICML
References in corpus (10)
- Soft Actor-Critic Algorithms and Applications
- Gaussian Process Optimization in the Bandit Setting: No Regret and Experimental Design
- Large-Scale Evolution of Image Classifiers
- Hindsight Experience Replay
- Efficient Architecture Search by Network Transformation
- Progressive Neural Architecture Search
- Evolution-Guided Policy Gradient in Reinforcement Learning
- Residual Policy Learning
- FollowNet: Robot Navigation by Following Natural Language Directions with Deep Reinforcement Learning
- Learning to Walk via Deep Reinforcement Learning
Cited by in corpus (9)
- Automated Reinforcement Learning (AutoRL): A Survey and Open Problems
- Learning to Seek: Autonomous Source Seeking with Deep Reinforcement Learning Onboard a Nano Drone Microcontroller
- Evolving Reinforcement Learning Algorithms
- Meta-learning curiosity algorithms
- Sample-Efficient Automated Deep Reinforcement Learning
- Neural Architecture Evolution in Deep Reinforcement Learning for Continuous Control
- A model-based framework for learning transparent swarm behaviors
- Learning to Win, Lose and Cooperate through Reward Signal Evolution
- Learning Robotic Manipulation Skills Using an Adaptive Force-Impedance Action Space