Point Transformer

Abstract

In this work, we present Point Transformer, a deep neural network that operates directly on unordered and unstructured point sets. We design Point Transformer to extract local and global features and relate both representations by introducing the local-global attention mechanism, which aims to capture spatial point relations and shape information. For that purpose, we propose SortNet, as part of the Point Transformer, which induces input permutation invariance by selecting points based on a learned score. The output of Point Transformer is a sorted and permutation invariant feature list that can directly be incorporated into common computer vision applications. We evaluate our approach on standard classification and part segmentation benchmarks to demonstrate competitive results compared to the prior work. Code is publicly available at: https://github.com/engelnico/point-transformer

References in corpus (10)

• PyTorch: An Imperative Style, High-Performance Deep Learning Library
• Neural Machine Translation by Jointly Learning to Align and Translate
• ShapeNet: An Information-Rich 3D Model Repository
• Linformer: Self-Attention with Linear Complexity
• On the Variance of the Adaptive Learning Rate and Beyond
• FPNN: Field Probing Neural Networks for 3D Data
• Nyströmformer: A Nyström-Based Algorithm for Approximating Self-Attention
• On the Limitations of Representing Functions on Sets
• DeepCLR: Correspondence-Less Architecture for Deep End-to-End Point Cloud Registration
• StickyPillars: Robust and Efficient Feature Matching on Point Clouds using Graph Neural Networks

Cited by in corpus (12)

• A Survey on Visual Transformer
• Deep Learning-based 3D Point Cloud Classification: A Systematic Survey and Outlook
• A comprehensive overview of deep learning techniques for 3D point cloud classification and semantic segmentation
• 6D-ViT: Category-Level 6D Object Pose Estimation via Transformer-based Instance Representation Learning
• Transformers for Multi-Object Tracking on Point Clouds
• Beyond Euclid: An Illustrated Guide to Modern Machine Learning with Geometric, Topological, and Algebraic Structures
• Benchmarking Deep Learning Architectures for Urban Vegetation Point Cloud Semantic Segmentation from MLS
• Harnessing Scale and Physics: A Multi-Graph Neural Operator Framework for PDEs on Arbitrary Geometries
• Hierarchical Point Cloud Encoding and Decoding with Lightweight Self-Attention based Model
• TP3M: Transformer-based Pseudo 3D Image Matching with Reference Image
• Pointy - A Lightweight Transformer for Point Cloud Foundation Models
• PCFEx: Point Cloud Feature Extraction for Graph Neural Networks