Picasso: A CUDA-based Library for Deep Learning over 3D Meshes

Abstract

We present Picasso, a CUDA-based library comprising novel modules for deep learning over complex real-world 3D meshes. Hierarchical neural architectures have proved effective in multi-scale feature extraction which signifies the need for fast mesh decimation. However, existing methods rely on CPU-based implementations to obtain multi-resolution meshes. We design GPU-accelerated mesh decimation to facilitate network resolution reduction efficiently on-the-fly. Pooling and unpooling modules are defined on the vertex clusters gathered during decimation. For feature learning over meshes, Picasso contains three types of novel convolutions namely, facet2vertex, vertex2facet, and facet2facet convolution. Hence, it treats a mesh as a geometric structure comprising vertices and facets, rather than a spatial graph with edges as previous methods do. Picasso also incorporates a fuzzy mechanism in its filters for robustness to mesh sampling (vertex density). It exploits Gaussian mixtures to define fuzzy coefficients for the facet2vertex convolution, and barycentric interpolation to define the coefficients for the remaining two convolutions. In this release, we demonstrate the effectiveness of the proposed modules with competitive segmentation results on S3DIS. The library will be made public through https://github.com/hlei-ziyan/Picasso.

arXiv commentAccepted to CVPR2021

References in corpus (8)

• Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift
• Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks
• Geometric deep learning: going beyond Euclidean data
• Instance Normalization: The Missing Ingredient for Fast Stylization
• Open3D: A Modern Library for 3D Data Processing
• MeshCNN: A Network with an Edge
• ScanNet: Richly-annotated 3D Reconstructions of Indoor Scenes
• Dynamic Graph CNN for Learning on Point Clouds