FPNN: Field Probing Neural Networks for 3D Data

Abstract

Building discriminative representations for 3D data has been an important task in computer graphics and computer vision research. Convolutional Neural Networks (CNNs) have shown to operate on 2D images with great success for a variety of tasks. Lifting convolution operators to 3D (3DCNNs) seems like a plausible and promising next step. Unfortunately, the computational complexity of 3D CNNs grows cubically with respect to voxel resolution. Moreover, since most 3D geometry representations are boundary based, occupied regions do not increase proportionately with the size of the discretization, resulting in wasted computation. In this work, we represent 3D spaces as volumetric fields, and propose a novel design that employs field probing filters to efficiently extract features from them. Each field probing filter is a set of probing points --- sensors that perceive the space. Our learning algorithm optimizes not only the weights associated with the probing points, but also their locations, which deforms the shape of the probing filters and adaptively distributes them in 3D space. The optimized probing points sense the 3D space "intelligently", rather than operating blindly over the entire domain. We show that field probing is significantly more efficient than 3DCNNs, while providing state-of-the-art performance, on classification tasks for 3D object recognition benchmark datasets.

To appear in NIPS 2016

References in corpus (2)

• Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift
• Caffe: Convolutional Architecture for Fast Feature Embedding

Cited by in corpus (37)

• PointNet: Deep Learning on Point Sets for 3D Classification and Segmentation
• MeshCNN: A Network with an Edge
• PointSIFT: A SIFT-like Network Module for 3D Point Cloud Semantic Segmentation
• Point Transformer
• Deep Learning-based 3D Point Cloud Classification: A Systematic Survey and Outlook
• PointCNN: Convolution On $\mathcal{X}$-Transformed Points
• Indoor Scene Understanding in 2.5/3D for Autonomous Agents: A Survey
• L2G Auto-encoder: Understanding Point Clouds by Local-to-Global Reconstruction with Hierarchical Self-Attention
• Advancements in Point Cloud-Based 3D Defect Detection and Classification for Industrial Systems: A Comprehensive Survey
• RotationNet: Joint Object Categorization and Pose Estimation Using Multiviews from Unsupervised Viewpoints
• Mining Point Cloud Local Structures by Kernel Correlation and Graph Pooling
• Tangent Convolutions for Dense Prediction in 3D
• Triplet-Center Loss for Multi-View 3D Object Retrieval
• Point2Sequence: Learning the Shape Representation of 3D Point Clouds with an Attention-based Sequence to Sequence Network
• H-CNN: Spatial Hashing Based CNN for 3D Shape Analysis
• Interpolated Convolutional Networks for 3D Point Cloud Understanding
• SurfNet: Generating 3D shape surfaces using deep residual networks
• Noise-resistant Deep Learning for Object Classification in 3D Point Clouds Using a Point Pair Descriptor
• Deep RBFNet: Point Cloud Feature Learning using Radial Basis Functions
• Distilling Knowledge from Graph Convolutional Networks
• Fully-Convolutional Point Networks for Large-Scale Point Clouds
• Spherical Convolutional Neural Network for 3D Point Clouds
• PointSCNet: Point Cloud Structure and Correlation Learning Based on Space Filling Curve-Guided Sampling
• Spherical Kernel for Efficient Graph Convolution on 3D Point Clouds
• Modeling Local Geometric Structure of 3D Point Clouds using Geo-CNN
• Grasping Field: Learning Implicit Representations for Human Grasps
• Semantic Segmentation of Urban Textured Meshes Through Point Sampling
• MeshNet: Mesh Neural Network for 3D Shape Representation
• Deep and Shallow Covariance Feature Quantization for 3D Facial Expression Recognition
• Multi-Resolution Graph Neural Network for Large-Scale Pointcloud Segmentation
• What can I do here? Leveraging Deep 3D saliency and geometry for fast and scalable multiple affordance detection
• Angular Triplet-Center Loss for Multi-view 3D Shape Retrieval
• A Survey on Deep Geometry Learning: From a Representation Perspective
• DV-ConvNet: Fully Convolutional Deep Learning on Point Clouds with Dynamic Voxelization and 3D Group Convolution
• LRC-Net: Learning Discriminative Features on Point Clouds by Encoding Local Region Contexts
• VERAM: View-Enhanced Recurrent Attention Model for 3D Shape Classification
• Rethinking PointNet Embedding for Faster and Compact Model