JSENet: Joint Semantic Segmentation and Edge Detection Network for 3D Point Clouds

Abstract

Semantic segmentation and semantic edge detection can be seen as two dual problems with close relationships in computer vision. Despite the fast evolution of learning-based 3D semantic segmentation methods, little attention has been drawn to the learning of 3D semantic edge detectors, even less to a joint learning method for the two tasks. In this paper, we tackle the 3D semantic edge detection task for the first time and present a new two-stream fully-convolutional network that jointly performs the two tasks. In particular, we design a joint refinement module that explicitly wires region information and edge information to improve the performances of both tasks. Further, we propose a novel loss function that encourages the network to produce semantic segmentation results with better boundaries. Extensive evaluations on S3DIS and ScanNet datasets show that our method achieves on par or better performance than the state-of-the-art methods for semantic segmentation and outperforms the baseline methods for semantic edge detection. Code release: https://github.com/hzykent/JSENet

arXiv commentAccepted to ECCV 2020, supplementary materials included

References in corpus (8)

• ScanNet: Richly-annotated 3D Reconstructions of Indoor Scenes
• Deep Parametric Continuous Convolutional Neural Networks
• Monte Carlo Convolution for Learning on Non-Uniformly Sampled Point Clouds
• Classification of Point Cloud Scenes with Multiscale Voxel Deep Network
• VV-Net: Voxel VAE Net with Group Convolutions for Point Cloud Segmentation
• Semantic Edge Detection with Diverse Deep Supervision
• Spherical Kernel for Efficient Graph Convolution on 3D Point Clouds
• Panoptic Edge Detection

Cited by in corpus (8)

• SegGroup: Seg-Level Supervision for 3D Instance and Semantic Segmentation
• GeoSegNet: Point Cloud Semantic Segmentation via Geometric Encoder-Decoder Modeling
• Dense Supervision Propagation for Weakly Supervised Semantic Segmentation on 3D Point Clouds
• Push-the-Boundary: Boundary-aware Feature Propagation for Semantic Segmentation of 3D Point Clouds
• One Thing One Click: A Self-Training Approach for Weakly Supervised 3D Semantic Segmentation
• Pri3D: Can 3D Priors Help 2D Representation Learning?
• PnP-3D: A Plug-and-Play for 3D Point Clouds
• Robust 3D Scene Segmentation through Hierarchical and Learnable Part-Fusion