Localization and Mapping using Instance-specific Mesh Models

Abstract

This paper focuses on building semantic maps, containing object poses and shapes, using a monocular camera. This is an important problem because robots need rich understanding of geometry and context if they are to shape the future of transportation, construction, and agriculture. Our contribution is an instance-specific mesh model of object shape that can be optimized online based on semantic information extracted from camera images. Multi-view constraints on the object shape are obtained by detecting objects and extracting category-specific keypoints and segmentation masks. We show that the errors between projections of the mesh model and the observed keypoints and masks can be differentiated in order to obtain accurate instance-specific object shapes. We evaluate the performance of the proposed approach in simulation and on the KITTI dataset by building maps of car poses and shapes.

8 pages, 9 figures

References in corpus (5)

• ORB-SLAM2: an Open-Source SLAM System for Monocular, Stereo and RGB-D Cameras
• VINS-Mono: A Robust and Versatile Monocular Visual-Inertial State Estimator
• Past, Present, and Future of Simultaneous Localization And Mapping: Towards the Robust-Perception Age
• Perspective Transformer Nets: Learning Single-View 3D Object Reconstruction without 3D Supervision
• FutureMapping: The Computational Structure of Spatial AI Systems

Cited by in corpus (3)

• Online Adaptation for Implicit Object Tracking and Shape Reconstruction in the Wild
• OrcVIO: Object residual constrained Visual-Inertial Odometry
• TerrainMesh: Metric-Semantic Terrain Reconstruction from Aerial Images Using Joint 2D-3D Learning