3D-MAN: 3D Multi-frame Attention Network for Object Detection

Abstract

3D object detection is an important module in autonomous driving and robotics. However, many existing methods focus on using single frames to perform 3D detection, and do not fully utilize information from multiple frames. In this paper, we present 3D-MAN: a 3D multi-frame attention network that effectively aggregates features from multiple perspectives and achieves state-of-the-art performance on Waymo Open Dataset. 3D-MAN first uses a novel fast single-frame detector to produce box proposals. The box proposals and their corresponding feature maps are then stored in a memory bank. We design a multi-view alignment and aggregation module, using attention networks, to extract and aggregate the temporal features stored in the memory bank. This effectively combines the features coming from different perspectives of the scene. We demonstrate the effectiveness of our approach on the large-scale complex Waymo Open Dataset, achieving state-of-the-art results compared to published single-frame and multi-frame methods.

References in corpus (6)

• Fast and Furious: Real Time End-to-End 3D Detection, Tracking and Motion Forecasting with a Single Convolutional Net
• IPOD: Intensive Point-based Object Detector for Point Cloud
• StarNet: Targeted Computation for Object Detection in Point Clouds
• From Points to Parts: 3D Object Detection from Point Cloud with Part-aware and Part-aggregation Network
• AFDet: Anchor Free One Stage 3D Object Detection
• STD: Sparse-to-Dense 3D Object Detector for Point Cloud