Recurrent Slice Networks for 3D Segmentation of Point Clouds

Abstract

Point clouds are an efficient data format for 3D data. However, existing 3D segmentation methods for point clouds either do not model local dependencies \cite{pointnet} or require added computations \cite{kd-net,pointnet2}. This work presents a novel 3D segmentation framework, RSNet\footnote{Codes are released here https://github.com/qianguih/RSNet}, to efficiently model local structures in point clouds. The key component of the RSNet is a lightweight local dependency module. It is a combination of a novel slice pooling layer, Recurrent Neural Network (RNN) layers, and a slice unpooling layer. The slice pooling layer is designed to project features of unordered points onto an ordered sequence of feature vectors so that traditional end-to-end learning algorithms (RNNs) can be applied. The performance of RSNet is validated by comprehensive experiments on the S3DIS\cite{stanford}, ScanNet\cite{scannet}, and ShapeNet \cite{shapenet} datasets. In its simplest form, RSNets surpass all previous state-of-the-art methods on these benchmarks. And comparisons against previous state-of-the-art methods \cite{pointnet, pointnet2} demonstrate the efficiency of RSNets.

arXiv commentcamera ready version for cvpr 2018 spotlight. codes are available here https://github.com/qianguih/RSNet

References in corpus (5)

• Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modeling
• ScanNet: Richly-annotated 3D Reconstructions of Indoor Scenes
• Spatially-sparse convolutional neural networks
• Predicting Depth, Surface Normals and Semantic Labels with a Common Multi-Scale Convolutional Architecture
• Sparse 3D convolutional neural networks

Cited by in corpus (10)

• RandLA-Net: Efficient Semantic Segmentation of Large-Scale Point Clouds
• MASC: Multi-scale Affinity with Sparse Convolution for 3D Instance Segmentation
• ShellNet: Efficient Point Cloud Convolutional Neural Networks using Concentric Shells Statistics
• Hierarchical Point-Edge Interaction Network for Point Cloud Semantic Segmentation
• Modeling Local Geometric Structure of 3D Point Clouds using Geo-CNN
• JSNet: Joint Instance and Semantic Segmentation of 3D Point Clouds
• Deep Functional Dictionaries: Learning Consistent Semantic Structures on 3D Models from Functions
• Object Segmentation of Cluttered Airborne LiDAR Point Clouds
• Deep Implicit Templates for 3D Shape Representation
• Multi Voxel-Point Neurons Convolution (MVPConv) for Fast and Accurate 3D Deep Learning