Point2Sequence: Learning the Shape Representation of 3D Point Clouds with an Attention-based Sequence to Sequence Network

Abstract

Exploring contextual information in the local region is important for shape understanding and analysis. Existing studies often employ hand-crafted or explicit ways to encode contextual information of local regions. However, it is hard to capture fine-grained contextual information in hand-crafted or explicit manners, such as the correlation between different areas in a local region, which limits the discriminative ability of learned features. To resolve this issue, we propose a novel deep learning model for 3D point clouds, named Point2Sequence, to learn 3D shape features by capturing fine-grained contextual information in a novel implicit way. Point2Sequence employs a novel sequence learning model for point clouds to capture the correlations by aggregating multi-scale areas of each local region with attention. Specifically, Point2Sequence first learns the feature of each area scale in a local region. Then, it captures the correlation between area scales in the process of aggregating all area scales using a recurrent neural network (RNN) based encoder-decoder structure, where an attention mechanism is proposed to highlight the importance of different area scales. Experimental results show that Point2Sequence achieves state-of-the-art performance in shape classification and segmentation tasks.

arXiv commentTo be published in AAAI 2019

References in corpus (9)

• Neural Machine Translation by Jointly Learning to Align and Translate
• Sequence to Sequence Learning with Neural Networks
• Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translation
• Effective Approaches to Attention-based Neural Machine Translation
• Dynamic Graph CNN for Learning on Point Clouds
• Multi-view Convolutional Neural Networks for 3D Shape Recognition
• FPNN: Field Probing Neural Networks for 3D Data
• PointCNN: Convolution On $\mathcal{X}$-Transformed Points
• RotationNet: Joint Object Categorization and Pose Estimation Using Multiviews from Unsupervised Viewpoints

Cited by in corpus (6)

• Patch-based Progressive 3D Point Set Upsampling
• SRINet: Learning Strictly Rotation-Invariant Representations for Point Cloud Classification and Segmentation
• Fast Hierarchical Neural Network for Feature Learning on Point Cloud
• A Sketch Based 3D Shape Retrieval Approach Based on Efficient Deep Point-to-Subspace Metric Learning
• Point Discriminative Learning for Data-efficient 3D Point Cloud Analysis
• Verifiability and Predictability: Interpreting Utilities of Network Architectures for Point Cloud Processing