Query-Efficient Imitation Learning for End-to-End Autonomous Driving
arXiv:1605.06450
Abstract
One way to approach end-to-end autonomous driving is to learn a policy function that maps from a sensory input, such as an image frame from a front-facing camera, to a driving action, by imitating an expert driver, or a reference policy. This can be done by supervised learning, where a policy function is tuned to minimize the difference between the predicted and ground-truth actions. A policy function trained in this way however is known to suffer from unexpected behaviours due to the mismatch between the states reachable by the reference policy and trained policy functions. More advanced algorithms for imitation learning, such as DAgger, addresses this issue by iteratively collecting training examples from both reference and trained policies. These algorithms often requires a large number of queries to a reference policy, which is undesirable as the reference policy is often expensive. In this paper, we propose an extension of the DAgger, called SafeDAgger, that is query-efficient and more suitable for end-to-end autonomous driving. We evaluate the proposed SafeDAgger in a car racing simulator and show that it indeed requires less queries to a reference policy. We observe a significant speed up in convergence, which we conjecture to be due to the effect of automated curriculum learning.
References in corpus (4)
Cited by in corpus (33)
- Motion Planning for Autonomous Driving: The State of the Art and Future Perspectives
- A Survey of End-to-End Driving: Architectures and Training Methods
- Testing Scenario Library Generation for Connected and Automated Vehicles, Part I: Methodology
- A Survey of Deep RL and IL for Autonomous Driving Policy Learning
- Virtual to Real Reinforcement Learning for Autonomous Driving
- InfoGAIL: Interpretable Imitation Learning from Visual Demonstrations
- Testing Scenario Library Generation for Connected and Automated Vehicles, Part II: Case Studies
- Active One-shot Learning
- Backplay: "Man muss immer umkehren"
- AC-Teach: A Bayesian Actor-Critic Method for Policy Learning with an Ensemble of Suboptimal Teachers
- The Game Imitation: Deep Supervised Convolutional Networks for Quick Video Game AI
- DropoutDAgger: A Bayesian Approach to Safe Imitation Learning
- Towards Safe Policy Improvement for Non-Stationary MDPs
- Affordance-based Reinforcement Learning for Urban Driving
- Meta learning Framework for Automated Driving
- Fighting Failures with FIRE: Failure Identification to Reduce Expert Burden in Intervention-Based Learning
- A Survey of Deep Reinforcement Learning Algorithms for Motion Planning and Control of Autonomous Vehicles
- Modular Vehicle Control for Transferring Semantic Information Between Weather Conditions Using GANs
- Safe Driving via Expert Guided Policy Optimization
- Avoidance of Manual Labeling in Robotic Autonomous Navigation Through Multi-Sensory Semi-Supervised Learning
- Human and Multi-Agent collaboration in a human-MARL teaming framework
- Comparing Human-Centric and Robot-Centric Sampling for Robot Deep Learning from Demonstrations
- Improving Learning from Demonstrations by Learning from Experience
- Offline Preference-Based Apprenticeship Learning
- Toward Imitating Visual Attention of Experts in Software Development Tasks
- School of hard knocks: Curriculum analysis for Pommerman with a fixed computational budget
- Behavior Planning at Urban Intersections through Hierarchical Reinforcement Learning
- Integrating Imitation Learning with Human Driving Data into Reinforcement Learning to Improve Training Efficiency for Autonomous Driving
- Iterative Imitation Policy Improvement for Interactive Autonomous Driving
- RadGrad: Active learning with loss gradients
- Autoregressive Knowledge Distillation through Imitation Learning
- On Memory Mechanism in Multi-Agent Reinforcement Learning
- Uncertainty-Aware Data Aggregation for Deep Imitation Learning