PCT: Point cloud transformer

Abstract

The irregular domain and lack of ordering make it challenging to design deep neural networks for point cloud processing. This paper presents a novel framework named Point Cloud Transformer(PCT) for point cloud learning. PCT is based on Transformer, which achieves huge success in natural language processing and displays great potential in image processing. It is inherently permutation invariant for processing a sequence of points, making it well-suited for point cloud learning. To better capture local context within the point cloud, we enhance input embedding with the support of farthest point sampling and nearest neighbor search. Extensive experiments demonstrate that the PCT achieves the state-of-the-art performance on shape classification, part segmentation and normal estimation tasks.

arXiv comment11 pages, 5 figures

References in corpus (2)

• BioBERT: a pre-trained biomedical language representation model for biomedical text mining
• Monte Carlo Convolution for Learning on Non-Uniformly Sampled Point Clouds

Cited by in corpus (71)

• A Survey on Visual Transformer
• Attention Mechanisms in Computer Vision: A Survey
• Radar-Camera Fusion for Object Detection and Semantic Segmentation in Autonomous Driving: A Comprehensive Review
• Comprehensive Review of Deep Learning-Based 3D Point Cloud Completion Processing and Analysis
• Deep Learning-based 3D Point Cloud Classification: A Systematic Survey and Outlook
• Subdivision-Based Mesh Convolution Networks
• Point Cloud Classification Using Content-based Transformer via Clustering in Feature Space
• A comprehensive overview of deep learning techniques for 3D point cloud classification and semantic segmentation
• Transformers in Single Object Tracking: An Experimental Survey
• Parameter-Efficient Person Re-identification in the 3D Space
• LACV-Net: Semantic Segmentation of Large-Scale Point Cloud Scene via Local Adaptive and Comprehensive VLAD
• Real-time 3D Single Object Tracking with Transformer
• GIFS: Neural Implicit Function for General Shape Representation
• 6D-ViT: Category-Level 6D Object Pose Estimation via Transformer-based Instance Representation Learning
• Laplacian2Mesh: Laplacian-Based Mesh Understanding
• CSDN: Cross-modal Shape-transfer Dual-refinement Network for Point Cloud Completion
• Vision-Language-Action Models for Robotics: A Review Towards Real-World Applications
• Point Cloud Transformers applied to Collider Physics
• Advancements in Point Cloud-Based 3D Defect Detection and Classification for Industrial Systems: A Comprehensive Survey
• Gaussian Radar Transformer for Semantic Segmentation in Noisy Radar Data
• MKConv: Multidimensional Feature Representation for Point Cloud Analysis
• PointDifformer: Robust Point Cloud Registration With Neural Diffusion and Transformer
• Leveraging Single-View Images for Unsupervised 3D Point Cloud Completion
• 3DMASC: Accessible, explainable 3D point clouds classification. Application to Bi-spectral Topo-bathymetric lidar data
• GeoSegNet: Point Cloud Semantic Segmentation via Geometric Encoder-Decoder Modeling
• Unbinned Deep Learning Jet Substructure Measurement in High $Q^2$ ep collisions at HERA
• Event Voxel Set Transformer for Spatiotemporal Representation Learning on Event Streams
• OBMO: One Bounding Box Multiple Objects for Monocular 3D Object Detection
• PointSCNet: Point Cloud Structure and Correlation Learning Based on Space Filling Curve-Guided Sampling
• Tinto: Multisensor Benchmark for 3D Hyperspectral Point Cloud Segmentation in the Geosciences
• Unsupervised deep learning for semantic segmentation of multispectral LiDAR forest point clouds
• Small but Mighty: Enhancing 3D Point Clouds Semantic Segmentation with U-Next Framework
• Convolution-enhanced Evolving Attention Networks
• Impact of color and mixing proportion of synthetic point clouds on semantic segmentation
• Regress Before Construct: Regress Autoencoder for Point Cloud Self-supervised Learning
• Probing the effects of broken symmetries in machine learning
• Beyond First Impressions: Integrating Joint Multi-modal Cues for Comprehensive 3D Representation
• iBA: Backdoor Attack on 3D Point Cloud via Reconstructing Itself
• Radar Velocity Transformer: Single-scan Moving Object Segmentation in Noisy Radar Point Clouds
• Persistent Homology Meets Object Unity: Object Recognition in Clutter
• Beyond Euclid: An Illustrated Guide to Modern Machine Learning with Geometric, Topological, and Algebraic Structures
• PointSmile: Point Self-supervised Learning via Curriculum Mutual Information
• On-the-fly Point Feature Representation for Point Clouds Analysis
• Point-DAE: Denoising Autoencoders for Self-supervised Point Cloud Learning
• HyperPointFormer: Multimodal Fusion in 3D Space with Dual-Branch Cross-Attention Transformers
• PotholeGuard: A Pothole Detection Approach by Point Cloud Semantic Segmentation
• Hierarchical Point Cloud Encoding and Decoding with Lightweight Self-Attention based Model
• DrivAer Transformer: A high-precision and fast prediction method for vehicle aerodynamic drag coefficient based on the DrivAerNet++ dataset
• ExpPoint-MAE: Better interpretability and performance for self-supervised point cloud transformers
• EdgeFormer: local patch-based edge detection transformer on point clouds
• TCFNet: Bidirectional face-bone transformation via a Transformer-based coarse-to-fine point movement network
• Self-supervised Learning of Rotation-invariant 3D Point Set Features using Transformer and its Self-distillation
• Neural Garment Dynamics via Manifold-Aware Transformers
• Normal Transformer: Extracting Surface Geometry from LiDAR Points Enhanced by Visual Semantics
• StreakNet-Arch: An Anti-scattering Network-based Architecture for Underwater Carrier LiDAR-Radar Imaging
• Radar Tracker: Moving Instance Tracking in Sparse and Noisy Radar Point Clouds
• Guiding Attention in End-to-End Driving Models
• Local region-learning modules for point cloud classification
• WHU-STree: A Multi-modal Benchmark Dataset for Street Tree Inventory
• Gap Completion in Point Cloud Scene occluded by Vehicles using SGC-Net
• To Make Yourself Invisible with Adversarial Semantic Contours
• Active search and coverage using point-cloud reinforcement learning
• No-reference geometry quality assessment for colorless point clouds via list-wise rank learning
• TP3M: Transformer-based Pseudo 3D Image Matching with Reference Image
• Density-aware global-local attention network for point cloud segmentation
• Joint Learning for Scattered Point Cloud Understanding with Hierarchical Self-Distillation
• Vision Transformers for End-to-End Quark-Gluon Jet Classification from Calorimeter Images
• A Survey on Deep Learning Architectures for Point Cloud Classification and Segmentation
• RWKV-PCSSC: Exploring RWKV Model for Point Cloud Semantic Scene Completion
• Pointy - A Lightweight Transformer for Point Cloud Foundation Models
• EuLearn: A 3D database for learning Euler characteristics