FusionNet: 3D Object Classification Using Multiple Data Representations

Abstract

High-quality 3D object recognition is an important component of many vision and robotics systems. We tackle the object recognition problem using two data representations, to achieve leading results on the Princeton ModelNet challenge. The two representations: 1. Volumetric representation: the 3D object is discretized spatially as binary voxels - $1$ if the voxel is occupied and $0$ otherwise. 2. Pixel representation: the 3D object is represented as a set of projected 2D pixel images. Current leading submissions to the ModelNet Challenge use Convolutional Neural Networks (CNNs) on pixel representations. However, we diverge from this trend and additionally, use Volumetric CNNs to bridge the gap between the efficiency of the above two representations. We combine both representations and exploit them to learn new features, which yield a significantly better classifier than using either of the representations in isolation. To do this, we introduce new Volumetric CNN (V-CNN) architectures.

References in corpus (5)

• Very Deep Convolutional Networks for Large-Scale Image Recognition
• Caffe: Convolutional Architecture for Fast Feature Embedding
• Going Deeper with Convolutions
• Rich feature hierarchies for accurate object detection and semantic segmentation
• Dimension Independent Matrix Square using MapReduce

Cited by in corpus (28)

• Indoor Scene Understanding in 2.5/3D for Autonomous Agents: A Survey
• A survey of synthetic data augmentation methods in computer vision
• Weakly-supervised DCNN for RGB-D Object Recognition in Real-World Applications Which Lack Large-scale Annotated Training Data
• SPLATNet: Sparse Lattice Networks for Point Cloud Processing
• Multi-View 3D Object Detection Network for Autonomous Driving
• RotationNet: Joint Object Categorization and Pose Estimation Using Multiviews from Unsupervised Viewpoints
• 3D Point Cloud Classification and Segmentation using 3D Modified Fisher Vector Representation for Convolutional Neural Networks
• 3D Object Instance Recognition and Pose Estimation Using Triplet Loss with Dynamic Margin
• PVNet: A Joint Convolutional Network of Point Cloud and Multi-View for 3D Shape Recognition
• A survey of Object Classification and Detection based on 2D/3D data
• Deep Learning Improves Template Matching by Normalized Cross Correlation
• Hybrid Bayesian Eigenobjects: Combining Linear Subspace and Deep Network Methods for 3D Robot Vision
• View Invariant Human Body Detection and Pose Estimation from Multiple Depth Sensors
• DualSDF: Semantic Shape Manipulation using a Two-Level Representation
• MeshNet: Mesh Neural Network for 3D Shape Representation
• LP-3DCNN: Unveiling Local Phase in 3D Convolutional Neural Networks
• PVRNet: Point-View Relation Neural Network for 3D Shape Recognition
• Discrete Rotation Equivariance for Point Cloud Recognition
• 3D Depthwise Convolution: Reducing Model Parameters in 3D Vision Tasks
• EnzyNet: enzyme classification using 3D convolutional neural networks on spatial representation
• 3DMaterialGAN: Learning 3D Shape Representation from Latent Space for Materials Science Applications
• VERAM: View-Enhanced Recurrent Attention Model for 3D Shape Classification
• A Deeper Look at 3D Shape Classifiers
• RocNet: Recursive Octree Network for Efficient 3D Deep Representation
• Accelerate 3D Object Processing via Spectral Layout
• Design, Analysis and Application of A Volumetric Convolutional Neural Network
• Human Recognition Using Face in Computed Tomography
• MV-C3D: A Spatial Correlated Multi-View 3D Convolutional Neural Networks