Action-Based Representation Learning for Autonomous Driving

Abstract

Human drivers produce a vast amount of data which could, in principle, be used to improve autonomous driving systems. Unfortunately, seemingly straightforward approaches for creating end-to-end driving models that map sensor data directly into driving actions are problematic in terms of interpretability, and typically have significant difficulty dealing with spurious correlations. Alternatively, we propose to use this kind of action-based driving data for learning representations. Our experiments show that an affordance-based driving model pre-trained with this approach can leverage a relatively small amount of weakly annotated imagery and outperform pure end-to-end driving models, while being more interpretable. Further, we demonstrate how this strategy outperforms previous methods based on learning inverse dynamics models as well as other methods based on heavy human supervision (ImageNet).

This paper has been accepted to the Conference on Robot Learning (CoRL 2020)

References in corpus (11)

• A Simple Framework for Contrastive Learning of Visual Representations
• End to End Learning for Self-Driving Cars
• A Survey of Autonomous Driving: Common Practices and Emerging Technologies
• A Survey of Deep Learning Techniques for Autonomous Driving
• Momentum Contrast for Unsupervised Visual Representation Learning
• Learning Latent Dynamics for Planning from Pixels
• Self-supervised Domain Adaptation for Computer Vision Tasks
• Causal Confusion in Imitation Learning
• Understanding intermediate layers using linear classifier probes
• Loss is its own Reward: Self-Supervision for Reinforcement Learning
• SAM: Squeeze-and-Mimic Networks for Conditional Visual Driving Policy Learning