SyncSpecCNN: Synchronized Spectral CNN for 3D Shape Segmentation

Abstract

In this paper, we study the problem of semantic annotation on 3D models that are represented as shape graphs. A functional view is taken to represent localized information on graphs, so that annotations such as part segment or keypoint are nothing but 0-1 indicator vertex functions. Compared with images that are 2D grids, shape graphs are irregular and non-isomorphic data structures. To enable the prediction of vertex functions on them by convolutional neural networks, we resort to spectral CNN method that enables weight sharing by parameterizing kernels in the spectral domain spanned by graph laplacian eigenbases. Under this setting, our network, named SyncSpecCNN, strive to overcome two key challenges: how to share coefficients and conduct multi-scale analysis in different parts of the graph for a single shape, and how to share information across related but different shapes that may be represented by very different graphs. Towards these goals, we introduce a spectral parameterization of dilated convolutional kernels and a spectral transformer network. Experimentally we tested our SyncSpecCNN on various tasks, including 3D shape part segmentation and 3D keypoint prediction. State-of-the-art performance has been achieved on all benchmark datasets.

References in corpus (4)

• Spectral Networks and Locally Connected Networks on Graphs
• Convolutional Neural Networks on Graphs with Fast Localized Spectral Filtering
• Deep Convolutional Networks on Graph-Structured Data
• Learning shape correspondence with anisotropic convolutional neural networks

Cited by in corpus (11)

• PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Space
• PointSIFT: A SIFT-like Network Module for 3D Point Cloud Semantic Segmentation
• SpectralNet: Spectral Clustering using Deep Neural Networks
• RGCNN: Regularized Graph CNN for Point Cloud Segmentation
• SPLATNet: Sparse Lattice Networks for Point Cloud Processing
• Large-Scale 3D Shape Reconstruction and Segmentation from ShapeNet Core55
• DensePoint: Learning Densely Contextual Representation for Efficient Point Cloud Processing
• Octree Generating Networks: Efficient Convolutional Architectures for High-resolution 3D Outputs
• Pointwise Rotation-Invariant Network with Adaptive Sampling and 3D Spherical Voxel Convolution
• NormalNet: Learning-based Normal Filtering for Mesh Denoising
• Learning Equivariant Representations