Improving Generalization of Transfer Learning Across Domains Using Spatio-Temporal Features in Autonomous Driving

Abstract

Practical learning-based autonomous driving models must be capable of generalizing learned behaviors from simulated to real domains, and from training data to unseen domains with unusual image properties. In this paper, we investigate transfer learning methods that achieve robustness to domain shifts by taking advantage of the invariance of spatio-temporal features across domains. In this paper, we propose a transfer learning method to improve generalization across domains via transfer of spatio-temporal features and salient data augmentation. Our model uses a CNN-LSTM network with Inception modules for image feature extraction. Our method runs in two phases: Phase 1 involves training on source domain data, while Phase 2 performs training on target domain data that has been supplemented by feature maps generated using the Phase 1 model. Our model significantly improves performance in unseen test cases for both simulation-to-simulation transfer as well as simulation-to-real transfer by up to +37.3\% in test accuracy and up to +40.8\% in steering angle prediction, compared to other SOTA methods across multiple datasets.

arXiv comment6 pages, 3 figures, 8 tables

References in corpus (10)

• Neural Machine Translation by Jointly Learning to Align and Translate
• GANs Trained by a Two Time-Scale Update Rule Converge to a Local Nash Equilibrium
• Multiple Object Recognition with Visual Attention
• Curriculum Domain Adaptation for Semantic Segmentation of Urban Scenes
• Multimodal End-to-End Autonomous Driving
• A2D2: Audi Autonomous Driving Dataset
• Virtual to Real Reinforcement Learning for Autonomous Driving
• End-to-End Deep Learning for Steering Autonomous Vehicles Considering Temporal Dependencies
• Virtual-to-Real-World Transfer Learning for Robots on Wilderness Trails
• Transferring Autonomous Driving Knowledge on Simulated and Real Intersections