3D Bounding Box Estimation Using Deep Learning and Geometry

Abstract

We present a method for 3D object detection and pose estimation from a single image. In contrast to current techniques that only regress the 3D orientation of an object, our method first regresses relatively stable 3D object properties using a deep convolutional neural network and then combines these estimates with geometric constraints provided by a 2D object bounding box to produce a complete 3D bounding box. The first network output estimates the 3D object orientation using a novel hybrid discrete-continuous loss, which significantly outperforms the L2 loss. The second output regresses the 3D object dimensions, which have relatively little variance compared to alternatives and can often be predicted for many object types. These estimates, combined with the geometric constraints on translation imposed by the 2D bounding box, enable us to recover a stable and accurate 3D object pose. We evaluate our method on the challenging KITTI object detection benchmark both on the official metric of 3D orientation estimation and also on the accuracy of the obtained 3D bounding boxes. Although conceptually simple, our method outperforms more complex and computationally expensive approaches that leverage semantic segmentation, instance level segmentation and flat ground priors and sub-category detection. Our discrete-continuous loss also produces state of the art results for 3D viewpoint estimation on the Pascal 3D+ dataset.

To appear in IEEE Conference on Computer Vision and Pattern Recognition (CVPR) 2017

References in corpus (7)

• Very Deep Convolutional Networks for Large-Scale Image Recognition
• Object Detectors Emerge in Deep Scene CNNs
• Rich feature hierarchies for accurate object detection and semantic segmentation
• Render for CNN: Viewpoint Estimation in Images Using CNNs Trained with Rendered 3D Model Views
• 6-DoF Object Pose from Semantic Keypoints
• Viewpoints and Keypoints
• 3D Object Class Detection in the Wild

Cited by in corpus (10)

• SINet: A Scale-insensitive Convolutional Neural Network for Fast Vehicle Detection
• Monocular 3D Object Detection with Pseudo-LiDAR Point Cloud
• SMOKE: Single-Stage Monocular 3D Object Detection via Keypoint Estimation
• A General Pipeline for 3D Detection of Vehicles
• A survey of Object Classification and Detection based on 2D/3D data
• Fusing Bird View LIDAR Point Cloud and Front View Camera Image for Deep Object Detection
• Scene Understanding Networks for Autonomous Driving based on Around View Monitoring System
• StarMap for Category-Agnostic Keypoint and Viewpoint Estimation
• Aug3D-RPN: Improving Monocular 3D Object Detection by Synthetic Images with Virtual Depth
• Learning to Predict the 3D Layout of a Scene