End-to-End Tracking and Semantic Segmentation Using Recurrent Neural Networks

Abstract

In this work we present a novel end-to-end framework for tracking and classifying a robot's surroundings in complex, dynamic and only partially observable real-world environments. The approach deploys a recurrent neural network to filter an input stream of raw laser measurements in order to directly infer object locations, along with their identity in both visible and occluded areas. To achieve this we first train the network using unsupervised Deep Tracking, a recently proposed theoretical framework for end-to-end space occupancy prediction. We show that by learning to track on a large amount of unsupervised data, the network creates a rich internal representation of its environment which we in turn exploit through the principle of inductive transfer of knowledge to perform the task of it's semantic classification. As a result, we show that only a small amount of labelled data suffices to steer the network towards mastering this additional task. Furthermore we propose a novel recurrent neural network architecture specifically tailored to tracking and semantic classification in real-world robotics applications. We demonstrate the tracking and classification performance of the method on real-world data collected at a busy road junction. Our evaluation shows that the proposed end-to-end framework compares favourably to a state-of-the-art, model-free tracking solution and that it outperforms a conventional one-shot training scheme for semantic classification.

References in corpus (4)

• Very Deep Convolutional Networks for Large-Scale Image Recognition
• Convolutional LSTM Network: A Machine Learning Approach for Precipitation Nowcasting
• On the difficulty of training Recurrent Neural Networks
• Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translation

Cited by in corpus (15)

• A Survey of Deep Learning Techniques for Autonomous Driving
• DeepIGeoS: A Deep Interactive Geodesic Framework for Medical Image Segmentation
• Uncertainty-aware Short-term Motion Prediction of Traffic Actors for Autonomous Driving
• Graph Neural Based End-to-end Data Association Framework for Online Multiple-Object Tracking
• Deep Tracking on the Move: Learning to Track the World from a Moving Vehicle using Recurrent Neural Networks
• Deep-Learned Collision Avoidance Policy for Distributed Multi-Agent Navigation
• Towards Lifelong Self-Supervision: A Deep Learning Direction for Robotics
• Deep Person Detection in 2D Range Data
• Autonomous Person-Specific Following Robot
• An unsupervised long short-term memory neural network for event detection in cell videos
• Dense CNN Learning with Equivalent Mappings
• An Efficient L-Shape Fitting Method for Vehicle Pose Detection with 2D LiDAR
• 2D LiDAR Map Prediction via Estimating Motion Flow with GRU
• DR-SPAAM: A Spatial-Attention and Auto-regressive Model for Person Detection in 2D Range Data
• Online Visual Robot Tracking and Identification using Deep LSTM Networks