PS^2-Net: A Locally and Globally Aware Network for Point-Based Semantic Segmentation

Abstract

In this paper, we present the PS^2-Net -- a locally and globally aware deep learning framework for semantic segmentation on 3D scene-level point clouds. In order to deeply incorporate local structures and global context to support 3D scene segmentation, our network is built on four repeatedly stacked encoders, where each encoder has two basic components: EdgeConv that captures local structures and NetVLAD that models global context. Different from existing start-of-the-art methods for point-based scene semantic segmentation that either violate or do not achieve permutation invariance, our PS^2-Net is designed to be permutation invariant which is an essential property of any deep network used to process unordered point clouds. We further provide theoretical proof to guarantee the permutation invariance property of our network. We perform extensive experiments on two large-scale 3D indoor scene datasets and demonstrate that our PS2-Net is able to achieve state-of-the-art performances as compared to existing approaches.

References in corpus (4)

• Fully Convolutional Networks for Semantic Segmentation
• All Models are Wrong, but Many are Useful: Learning a Variable's Importance by Studying an Entire Class of Prediction Models Simultaneously
• Generative and Discriminative Voxel Modeling with Convolutional Neural Networks
• Understanding Global Feature Contributions With Additive Importance Measures

Cited by in corpus (5)

• Interpretable Machine Learning -- A Brief History, State-of-the-Art and Challenges
• Enhance to Read Better: A Multi-Task Adversarial Network for Handwritten Document Image Enhancement
• Advanced Underwater Image Restoration in Complex Illumination Conditions
• Exploiting auto-encoders and segmentation methods for middle-level explanations of image classification systems
• Explanations Based on Item Response Theory (eXirt): A Model-Specific Method to Explain Tree-Ensemble Model in Trust Perspective