Learning Propagation for Arbitrarily-structured Data

Abstract

Processing an input signal that contains arbitrary structures, e.g., superpixels and point clouds, remains a big challenge in computer vision. Linear diffusion, an effective model for image processing, has been recently integrated with deep learning algorithms. In this paper, we propose to learn pairwise relations among data points in a global fashion to improve semantic segmentation with arbitrarily-structured data, through spatial generalized propagation networks (SGPN). The network propagates information on a group of graphs, which represent the arbitrarily-structured data, through a learned, linear diffusion process. The module is flexible to be embedded and jointly trained with many types of networks, e.g., CNNs. We experiment with semantic segmentation networks, where we use our propagation module to jointly train on different data -- images, superpixels and point clouds. We show that SGPN consistently improves the performance of both pixel and point cloud segmentation, compared to networks that do not contain this module. Our method suggests an effective way to model the global pairwise relations for arbitrarily-structured data.

arXiv commentICCV 2019

References in corpus (10)

• Encoder-Decoder with Atrous Separable Convolution for Semantic Image Segmentation
• Efficient Inference in Fully Connected CRFs with Gaussian Edge Potentials
• Conditional Random Fields as Recurrent Neural Networks
• The Cityscapes Dataset for Semantic Urban Scene Understanding
• Layer Normalization
• Wider or Deeper: Revisiting the ResNet Model for Visual Recognition
• Learning Deep Structured Models
• A Simple Baseline Algorithm for Graph Classification
• Tangent Convolutions for Dense Prediction in 3D
• Fast End-to-End Trainable Guided Filter