PointFusion: Deep Sensor Fusion for 3D Bounding Box Estimation
arXiv:1711.10871
Abstract
We present PointFusion, a generic 3D object detection method that leverages both image and 3D point cloud information. Unlike existing methods that either use multi-stage pipelines or hold sensor and dataset-specific assumptions, PointFusion is conceptually simple and application-agnostic. The image data and the raw point cloud data are independently processed by a CNN and a PointNet architecture, respectively. The resulting outputs are then combined by a novel fusion network, which predicts multiple 3D box hypotheses and their confidences, using the input 3D points as spatial anchors. We evaluate PointFusion on two distinctive datasets: the KITTI dataset that features driving scenes captured with a lidar-camera setup, and the SUN-RGBD dataset that captures indoor environments with RGB-D cameras. Our model is the first one that is able to perform better or on-par with the state-of-the-art on these diverse datasets without any dataset-specific model tuning.
arXiv commentCVPR 2018
References in corpus (13)
- Deep Residual Learning for Image Recognition
- PointNet: Deep Learning on Point Sets for 3D Classification and Segmentation
- Focal Loss for Dense Object Detection
- Feature Pyramid Networks for Object Detection
- DenseBox: Unifying Landmark Localization with End to End Object Detection
- Vehicle Detection from 3D Lidar Using Fully Convolutional Network
- Speed/accuracy trade-offs for modern convolutional object detectors
- Multi-View 3D Object Detection Network for Autonomous Driving
- Deep Sliding Shapes for Amodal 3D Object Detection in RGB-D Images
- Scalable Object Detection using Deep Neural Networks
- Multi-modal Sensor Registration for Vehicle Perception via Deep Neural Networks
- Viewpoints and Keypoints
- 3D Fully Convolutional Network for Vehicle Detection in Point Cloud
Cited by in corpus (7)
- Class-balanced Grouping and Sampling for Point Cloud 3D Object Detection
- PointRCNN: 3D Object Proposal Generation and Detection from Point Cloud
- 3D Object Detection Using Scale Invariant and Feature Reweighting Networks
- Robust Deep Multi-modal Learning Based on Gated Information Fusion Network
- View Invariant Human Body Detection and Pose Estimation from Multiple Depth Sensors
- Class-specific Anchoring Proposal for 3D Object Recognition in LIDAR and RGB Images
- Learning to Predict the 3D Layout of a Scene