Multi Voxel-Point Neurons Convolution (MVPConv) for Fast and Accurate 3D Deep Learning

Abstract

We present a new convolutional neural network, called Multi Voxel-Point Neurons Convolution (MVPConv), for fast and accurate 3D deep learning. The previous works adopt either individual point-based features or local-neighboring voxel-based features to process 3D model, which limits the performance of models due to the inefficient computation. Moreover, most of the existing 3D deep learning frameworks aim at solving one specific task, and only a few of them can handle a variety of tasks. Integrating both the advantages of the voxel and point-based methods, the proposed MVPConv can effectively increase the neighboring collection between point-based features and also promote the independence among voxel-based features. Simply replacing the corresponding convolution module with MVPConv, we show that MVPConv can fit in different backbones to solve a wide range of 3D tasks. Extensive experiments on benchmark datasets such as ShapeNet Part, S3DIS and KITTI for various tasks show that MVPConv improves the accuracy of the backbone (PointNet) by up to 36%, and achieves higher accuracy than the voxel-based model with up to 34 times speedup. In addition, MVPConv also outperforms the state-of-the-art point-based models with up to 8 times speedup. Notably, our MVPConv achieves better accuracy than the newest point-voxel-based model PVCNN (a model more efficient than PointNet) with lower latency.

References in corpus (11)

• Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift
• Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks
• Joint 2D-3D-Semantic Data for Indoor Scene Understanding
• Relation-Shape Convolutional Neural Network for Point Cloud Analysis
• SpiderCNN: Deep Learning on Point Sets with Parameterized Convolutional Filters
• SPLATNet: Sparse Lattice Networks for Point Cloud Processing
• Recurrent Slice Networks for 3D Segmentation of Point Clouds
• Tangent Convolutions for Dense Prediction in 3D
• PV-RCNN++: Point-Voxel Feature Set Abstraction With Local Vector Representation for 3D Object Detection
• VoxSegNet: Volumetric CNNs for Semantic Part Segmentation of 3D Shapes
• Self-Prediction for Joint Instance and Semantic Segmentation of Point Clouds