CMRNet: Camera to LiDAR-Map Registration

Abstract

In this paper we present CMRNet, a realtime approach based on a Convolutional Neural Network to localize an RGB image of a scene in a map built from LiDAR data. Our network is not trained in the working area, i.e. CMRNet does not learn the map. Instead it learns to match an image to the map. We validate our approach on the KITTI dataset, processing each frame independently without any tracking procedure. CMRNet achieves 0.27m and 1.07deg median localization accuracy on the sequence 00 of the odometry dataset, starting from a rough pose estimate displaced up to 3.5m and 17deg. To the best of our knowledge this is the first CNN-based approach that learns to match images from a monocular camera to a given, preexisting 3D LiDAR-map.

arXiv commentAccepted for presentation at IEEE ITSC2019

References in corpus (2)

• VLocNet++: Deep Multitask Learning for Semantic Visual Localization and Odometry
• Deconvolutional Networks for Point-Cloud Vehicle Detection and Tracking in Driving Scenarios

Cited by in corpus (9)

• Flood severity mapping from Volunteered Geographic Information by interpreting water level from images containing people: a case study of Hurricane Harvey
• Automatic Target-Less Camera-LiDAR Calibration From Motion and Deep Point Correspondences
• CMRNet++: Map and Camera Agnostic Monocular Visual Localization in LiDAR Maps
• RaLF: Flow-based Global and Metric Radar Localization in LiDAR Maps
• An experimental evaluation of Siamese Neural Networks for robot localization using omnidirectional imaging in indoor environments
• Tactile-based Exploration, Mapping and Navigation with Collision-Resilient Aerial Vehicles
• Monocular Camera Localization in Prior LiDAR Maps with 2D-3D Line Correspondences
• Self-localization on a 3D map by fusing global and local features from a monocular camera
• Improving GNSS Positioning using Neural Network-based Corrections