End-to-end Learning of Multi-sensor 3D Tracking by Detection

Abstract

In this paper we propose a novel approach to tracking by detection that can exploit both cameras as well as LIDAR data to produce very accurate 3D trajectories. Towards this goal, we formulate the problem as a linear program that can be solved exactly, and learn convolutional networks for detection as well as matching in an end-to-end manner. We evaluate our model in the challenging KITTI dataset and show very competitive results.

arXiv commentPresented at IEEE International Conference on Robotics and Automation (ICRA), 2018

References in corpus (4)

• Adam: A Method for Stochastic Optimization
• Very Deep Convolutional Networks for Large-Scale Image Recognition
• Computing the Stereo Matching Cost with a Convolutional Neural Network
• Multi-View 3D Object Detection Network for Autonomous Driving

Cited by in corpus (32)

• Lidar for Autonomous Driving: The principles, challenges, and trends for automotive lidar and perception systems
• Deep Learning for Image and Point Cloud Fusion in Autonomous Driving: A Review
• Invisible for both Camera and LiDAR: Security of Multi-Sensor Fusion based Perception in Autonomous Driving Under Physical-World Attacks
• DeepFusionMOT: A 3D Multi-Object Tracking Framework Based on Camera-LiDAR Fusion with Deep Association
• Semantics for Robotic Mapping, Perception and Interaction: A Survey
• A Bayesian Filter for Multi-view 3D Multi-object Tracking with Occlusion Handling
• 3D-SiamRPN: An End-to-End Learning Method for Real-Time 3D Single Object Tracking Using Raw Point Cloud
• Joint Monocular 3D Vehicle Detection and Tracking
• 3D Multi-Object Tracking: A Baseline and New Evaluation Metrics
• Autonomous Driving with Deep Learning: A Survey of State-of-Art Technologies
• DeepSignals: Predicting Intent of Drivers Through Visual Signals
• Key-Point Sequence Lossless Compression for Intelligent Video Analysis
• TrackMPNN: A Message Passing Graph Neural Architecture for Multi-Object Tracking
• PNAS-MOT: Multi-Modal Object Tracking with Pareto Neural Architecture Search
• PTP: Parallelized Tracking and Prediction with Graph Neural Networks and Diversity Sampling
• GNN3DMOT: Graph Neural Network for 3D Multi-Object Tracking with Multi-Feature Learning
• BLVD: Building A Large-scale 5D Semantics Benchmark for Autonomous Driving
• Robust Multi-Modality Multi-Object Tracking
• FlowMOT: 3D Multi-Object Tracking by Scene Flow Association
• PnPNet: End-to-End Perception and Prediction with Tracking in the Loop
• SoDA: Multi-Object Tracking with Soft Data Association
• Drift with Devil: Security of Multi-Sensor Fusion based Localization in High-Level Autonomous Driving under GPS Spoofing (Extended Version)
• End-to-end Recurrent Multi-Object Tracking and Trajectory Prediction with Relational Reasoning
• EagerMOT: 3D Multi-Object Tracking via Sensor Fusion
• Efficient and accurate object detection with simultaneous classification and tracking
• PointTrackNet: An End-to-End Network For 3-D Object Detection and Tracking From Point Clouds
• AutoSelect: Automatic and Dynamic Detection Selection for 3D Multi-Object Tracking
• Tracking from Patterns: Learning Corresponding Patterns in Point Clouds for 3D Object Tracking
• 3D Object Detection and Tracking Based on Streaming Data
• Ego-motion and Surrounding Vehicle State Estimation Using a Monocular Camera
• "Who is Driving around Me?" Unique Vehicle Instance Classification using Deep Neural Features
• Occlusion Aware Kernel Correlation Filter Tracker using RGB-D