RTM3D: Real-time Monocular 3D Detection from Object Keypoints for Autonomous Driving

Abstract

In this work, we propose an efficient and accurate monocular 3D detection framework in single shot. Most successful 3D detectors take the projection constraint from the 3D bounding box to the 2D box as an important component. Four edges of a 2D box provide only four constraints and the performance deteriorates dramatically with the small error of the 2D detector. Different from these approaches, our method predicts the nine perspective keypoints of a 3D bounding box in image space, and then utilize the geometric relationship of 3D and 2D perspectives to recover the dimension, location, and orientation in 3D space. In this method, the properties of the object can be predicted stably even when the estimation of keypoints is very noisy, which enables us to obtain fast detection speed with a small architecture. Training our method only uses the 3D properties of the object without the need for external networks or supervision data. Our method is the first real-time system for monocular image 3D detection while achieves state-of-the-art performance on the KITTI benchmark. Code will be released at https://github.com/Banconxuan/RTM3D.

11 pages, 4 figures and 7 tables

References in corpus (4)

• Very Deep Convolutional Networks for Large-Scale Image Recognition
• FCOS: Fully Convolutional One-Stage Object Detection
• Vehicle Detection from 3D Lidar Using Fully Convolutional Network
• Mono3D++: Monocular 3D Vehicle Detection with Two-Scale 3D Hypotheses and Task Priors

Cited by in corpus (19)

• AFDet: Anchor Free One Stage 3D Object Detection
• Stereo CenterNet based 3D Object Detection for Autonomous Driving
• Monocular 3D Object Detection with Sequential Feature Association and Depth Hint Augmentation
• Autonomous Driving with Deep Learning: A Survey of State-of-Art Technologies
• Attention-Based Depth Distillation with 3D-Aware Positional Encoding for Monocular 3D Object Detection
• Advancing Self-supervised Monocular Depth Learning with Sparse LiDAR
• Geometry Uncertainty Projection Network for Monocular 3D Object Detection
• Monocular 3D Detection with Geometric Constraints Embedding and Semi-supervised Training
• OCM3D: Object-Centric Monocular 3D Object Detection
• Lidar Point Cloud Guided Monocular 3D Object Detection
• Ground-aware Monocular 3D Object Detection for Autonomous Driving
• YOLOStereo3D: A Step Back to 2D for Efficient Stereo 3D Detection
• Aug3D-RPN: Improving Monocular 3D Object Detection by Synthetic Images with Virtual Depth
• IPS300+: a Challenging Multimodal Dataset for Intersection Perception System
• M3DSSD: Monocular 3D Single Stage Object Detector
• Lite-FPN for Keypoint-based Monocular 3D Object Detection
• UrbanNet: Leveraging Urban Maps for Long Range 3D Object Detection
• Efficient Online Transfer Learning for 3D Object Classification in Autonomous Driving
• IAFA: Instance-aware Feature Aggregation for 3D Object Detection from a Single Image