Gradient-free Policy Architecture Search and Adaptation
arXiv:1710.05958
Abstract
We develop a method for policy architecture search and adaptation via gradient-free optimization which can learn to perform autonomous driving tasks. By learning from both demonstration and environmental reward we develop a model that can learn with relatively few early catastrophic failures. We first learn an architecture of appropriate complexity to perceive aspects of world state relevant to the expert demonstration, and then mitigate the effect of domain-shift during deployment by adapting a policy demonstrated in a source domain to rewards obtained in a target environment. We show that our approach allows safer learning than baseline methods, offering a reduced cumulative crash metric over the agent's lifetime as it learns to drive in a realistic simulated environment.
arXiv commentAccepted in Conference on Robot Learning, 2017
References in corpus (8)
- Adam: A Method for Stochastic Optimization
- Deep Learning in Neural Networks: An Overview
- Neural Architecture Search with Reinforcement Learning
- End to End Learning for Self-Driving Cars
- Theoretical Models of Learning to Learn
- Evolution Strategies as a Scalable Alternative to Reinforcement Learning
- DeepDriving: Learning Affordance for Direct Perception in Autonomous Driving
- End-to-end Learning of Driving Models from Large-scale Video Datasets
Cited by in corpus (6)
- On the Relationship Between the OpenAI Evolution Strategy and Stochastic Gradient Descent
- Constrained-Space Optimization and Reinforcement Learning for Complex Tasks
- Adversarial Representation Active Learning
- Is it Safe to Drive? An Overview of Factors, Challenges, and Datasets for Driveability Assessment in Autonomous Driving
- Sparse Perturbations for Improved Convergence in Stochastic Zeroth-Order Optimization
- State-Relabeling Adversarial Active Learning