HDNET: Exploiting HD Maps for 3D Object Detection

Abstract

In this paper we show that High-Definition (HD) maps provide strong priors that can boost the performance and robustness of modern 3D object detectors. Towards this goal, we design a single stage detector that extracts geometric and semantic features from the HD maps. As maps might not be available everywhere, we also propose a map prediction module that estimates the map on the fly from raw LiDAR data. We conduct extensive experiments on KITTI as well as a large-scale 3D detection benchmark containing 1 million frames, and show that the proposed map-aware detector consistently outperforms the state-of-the-art in both mapped and un-mapped scenarios. Importantly the whole framework runs at 20 frames per second.

Spotlight paper at 2nd Conference on Robot Learning (CoRL 2018)

Cited by in corpus (66)

• Deep Multi-modal Object Detection and Semantic Segmentation for Autonomous Driving: Datasets, Methods, and Challenges
• PV-RCNN: Point-Voxel Feature Set Abstraction for 3D Object Detection
• PointRCNN: 3D Object Proposal Generation and Detection from Point Cloud
• Argoverse: 3D Tracking and Forecasting with Rich Maps
• COOPERNAUT: End-to-End Driving with Cooperative Perception for Networked Vehicles
• Pseudo-LiDAR++: Accurate Depth for 3D Object Detection in Autonomous Driving
• LookOut: Diverse Multi-Future Prediction and Planning for Self-Driving
• Semantics for Robotic Mapping, Perception and Interaction: A Survey
• StarNet: Targeted Computation for Object Detection in Point Clouds
• From Points to Parts: 3D Object Detection from Point Cloud with Part-aware and Part-aggregation Network
• End-to-End Multi-View Fusion for 3D Object Detection in LiDAR Point Clouds
• Occupancy Flow Fields for Motion Forecasting in Autonomous Driving
• Spatially-Aware Graph Neural Networks for Relational Behavior Forecasting from Sensor Data
• Object DGCNN: 3D Object Detection using Dynamic Graphs
• Deep Learning for 3D Point Clouds: A Survey
• RefinedMPL: Refined Monocular PseudoLiDAR for 3D Object Detection in Autonomous Driving
• 3D Multi-Object Tracking: A Baseline and New Evaluation Metrics
• Checkmate: Breaking the Memory Wall with Optimal Tensor Rematerialization
• Learning Lane Graph Representations for Motion Forecasting
• Monocular 3D Object Detection with Pseudo-LiDAR Point Cloud
• 3D IoU-Net: IoU Guided 3D Object Detector for Point Clouds
• PV-RCNN++: Point-Voxel Feature Set Abstraction With Local Vector Representation for 3D Object Detection
• TANet: Robust 3D Object Detection from Point Clouds with Triple Attention
• Pillar-based Object Detection for Autonomous Driving
• Auto4D: Learning to Label 4D Objects from Sequential Point Clouds
• HVNet: Hybrid Voxel Network for LiDAR Based 3D Object Detection
• PseudoAugment: Learning to Use Unlabeled Data for Data Augmentation in Point Clouds
• Identifying Unknown Instances for Autonomous Driving
• Associate-3Ddet: Perceptual-to-Conceptual Association for 3D Point Cloud Object Detection
• Deformable Filter Convolution for Point Cloud Reasoning
• Discrete Residual Flow for Probabilistic Pedestrian Behavior Prediction
• What You See is What You Get: Exploiting Visibility for 3D Object Detection
• Improving 3D Object Detection through Progressive Population Based Augmentation
• Depth-conditioned Dynamic Message Propagation for Monocular 3D Object Detection
• Active Perception using Light Curtains for Autonomous Driving
• Improving 3D Object Detection with Channel-wise Transformer
• SSN: Shape Signature Networks for Multi-class Object Detection from Point Clouds
• MotionNet: Joint Perception and Motion Prediction for Autonomous Driving Based on Bird's Eye View Maps
• End-to-End Pseudo-LiDAR for Image-Based 3D Object Detection
• FusionPainting: Multimodal Fusion with Adaptive Attention for 3D Object Detection
• Multi-Task Multi-Sensor Fusion for 3D Object Detection
• 3D for Free: Crossmodal Transfer Learning using HD Maps
• PnPNet: End-to-End Perception and Prediction with Tracking in the Loop
• StrObe: Streaming Object Detection from LiDAR Packets
• PolarStream: Streaming Lidar Object Detection and Segmentation with Polar Pillars
• S3CNet: A Sparse Semantic Scene Completion Network for LiDAR Point Clouds
• S-AT GCN: Spatial-Attention Graph Convolution Network based Feature Enhancement for 3D Object Detection
• Streaming Object Detection for 3-D Point Clouds
• YOLOStereo3D: A Step Back to 2D for Efficient Stereo 3D Detection
• 3D Object Detection From LiDAR Data Using Distance Dependent Feature Extraction
• Geometry-Aware Instance Segmentation with Disparity Maps
• Review: deep learning on 3D point clouds
• Safety-Oriented Pedestrian Motion and Scene Occupancy Forecasting
• Pseudo-labeling for Scalable 3D Object Detection
• Trust, but Verify: Cross-Modality Fusion for HD Map Change Detection
• Investigating Attention Mechanism in 3D Point Cloud Object Detection
• Uncertainty-Aware Vehicle Orientation Estimation for Joint Detection-Prediction Models
• Stereo RGB and Deeper LIDAR Based Network for 3D Object Detection
• UrbanNet: Leveraging Urban Maps for Long Range 3D Object Detection
• Unsupervised Object Detection with LiDAR Clues
• CubifAE-3D: Monocular Camera Space Cubification for Auto-Encoder based 3D Object Detection
• Boundary-Aware Dense Feature Indicator for Single-Stage 3D Object Detection from Point Clouds
• Context-Aware Dynamic Feature Extraction for 3D Object Detection in Point Clouds
• DV-Det: Efficient 3D Point Cloud Object Detection with Dynamic Voxelization
• All-Weather Object Recognition Using Radar and Infrared Sensing
• DAGMapper: Learning to Map by Discovering Lane Topology