CubeSLAM: Monocular 3D Object SLAM
arXiv:1806.00557 · doi:10.1109/TRO.2019.2909168
Abstract
We present a method for single image 3D cuboid object detection and multi-view object SLAM in both static and dynamic environments, and demonstrate that the two parts can improve each other. Firstly for single image object detection, we generate high-quality cuboid proposals from 2D bounding boxes and vanishing points sampling. The proposals are further scored and selected based on the alignment with image edges. Secondly, multi-view bundle adjustment with new object measurements is proposed to jointly optimize poses of cameras, objects and points. Objects can provide long-range geometric and scale constraints to improve camera pose estimation and reduce monocular drift. Instead of treating dynamic regions as outliers, we utilize object representation and motion model constraints to improve the camera pose estimation. The 3D detection experiments on SUN RGBD and KITTI show better accuracy and robustness over existing approaches. On the public TUM, KITTI odometry and our own collected datasets, our SLAM method achieves the state-of-the-art monocular camera pose estimation and at the same time, improves the 3D object detection accuracy.
arXiv commentIEEE Transactions on Robotics
References in corpus (5)
- ORB-SLAM: a Versatile and Accurate Monocular SLAM System
- High-Speed Tracking with Kernelized Correlation Filters
- DynaSLAM: Tracking, Mapping and Inpainting in Dynamic Scenes
- DeepVO: Towards End-to-End Visual Odometry with Deep Recurrent Convolutional Neural Networks
- Robust Dense Mapping for Large-Scale Dynamic Environments
Cited by in corpus (27)
- Perception and Navigation in Autonomous Systems in the Era of Learning: A Survey
- EAO-SLAM: Monocular Semi-Dense Object SLAM Based on Ensemble Data Association
- Towards Real-time Semantic RGB-D SLAM in Dynamic Environments
- Large-scale Autonomous Flight with Real-time Semantic SLAM under Dense Forest Canopy
- Monocular Object and Plane SLAM in Structured Environments
- AirDOS: Dynamic SLAM benefits from Articulated Objects
- From SLAM to Situational Awareness: Challenges and Survey
- Stereo CenterNet based 3D Object Detection for Autonomous Driving
- TwistSLAM: Constrained SLAM in Dynamic Environment
- An Object SLAM Framework for Association, Mapping, and High-Level Tasks
- A Bearing-Angle Approach for Unknown Target Motion Analysis Based on Visual Measurements
- 3D Active Metric-Semantic SLAM
- Active Metric-Semantic Mapping by Multiple Aerial Robots
- RO-MAP: Real-Time Multi-Object Mapping with Neural Radiance Fields
- Object SLAM-Based Active Mapping and Robotic Grasping
- GOReloc: Graph-based Object-Level Relocalization for Visual SLAM
- Object-Based Visual Camera Pose Estimation From Ellipsoidal Model and 3D-Aware Ellipse Prediction
- Monocular visual simultaneous localization and mapping: (r)evolution from geometry to deep learning-based pipelines
- Evaluating the Impact of Semantic Segmentation and Pose Estimation on Dense Semantic SLAM
- ELLIPSDF: Joint Object Pose and Shape Optimization with a Bi-level Ellipsoid and Signed Distance Function Description
- Multimotion Visual Odometry (MVO)
- CLIP-Clique: Graph-based Correspondence Matching Augmented by Vision Language Models for Object-based Global Localization
- Aerial Field Robotics
- Temporally Consistent Object 6D Pose Estimation for Robot Control
- ATG-PVD: Ticketing Parking Violations on A Drone
- Semantic Exploration and Dense Mapping of Complex Environments using Ground Robot with Panoramic LiDAR-Camera Fusion
- MSG-Loc: Multi-Label Likelihood-based Semantic Graph Matching for Object-Level Global Localization