3D Object Detection with Pointformer

Abstract

Feature learning for 3D object detection from point clouds is very challenging due to the irregularity of 3D point cloud data. In this paper, we propose Pointformer, a Transformer backbone designed for 3D point clouds to learn features effectively. Specifically, a Local Transformer module is employed to model interactions among points in a local region, which learns context-dependent region features at an object level. A Global Transformer is designed to learn context-aware representations at the scene level. To further capture the dependencies among multi-scale representations, we propose Local-Global Transformer to integrate local features with global features from higher resolution. In addition, we introduce an efficient coordinate refinement module to shift down-sampled points closer to object centroids, which improves object proposal generation. We use Pointformer as the backbone for state-of-the-art object detection models and demonstrate significant improvements over original models on both indoor and outdoor datasets.

arXiv commentAccepted by IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) 2021. Code is available at https://github.com/Vladimir2506/Pointformer

References in corpus (11)

• An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale
• PointNet: Deep Learning on Point Sets for 3D Classification and Segmentation
• Deformable DETR: Deformable Transformers for End-to-End Object Detection
• Linformer: Self-Attention with Linear Complexity
• ScanNet: Richly-annotated 3D Reconstructions of Indoor Scenes
• Reformer: The Efficient Transformer
• Class-balanced Grouping and Sampling for Point Cloud 3D Object Detection
• Transformers are RNNs: Fast Autoregressive Transformers with Linear Attention
• Rethinking Attention with Performers
• Point-GNN: Graph Neural Network for 3D Object Detection in a Point Cloud
• HVNet: Hybrid Voxel Network for LiDAR Based 3D Object Detection

Cited by in corpus (5)

• PPT Fusion: Pyramid Patch Transformerfor a Case Study in Image Fusion
• Improving 3D Object Detection with Channel-wise Transformer
• Voxel Transformer for 3D Object Detection
• Improved Pillar with Fine-grained Feature for 3D Object Detection
• Shape Prior Non-Uniform Sampling Guided Real-time Stereo 3D Object Detection