O-CNN: Octree-based Convolutional Neural Networks for 3D Shape Analysis

Abstract

We present O-CNN, an Octree-based Convolutional Neural Network (CNN) for 3D shape analysis. Built upon the octree representation of 3D shapes, our method takes the average normal vectors of a 3D model sampled in the finest leaf octants as input and performs 3D CNN operations on the octants occupied by the 3D shape surface. We design a novel octree data structure to efficiently store the octant information and CNN features into the graphics memory and execute the entire O-CNN training and evaluation on the GPU. O-CNN supports various CNN structures and works for 3D shapes in different representations. By restraining the computations on the octants occupied by 3D surfaces, the memory and computational costs of the O-CNN grow quadratically as the depth of the octree increases, which makes the 3D CNN feasible for high-resolution 3D models. We compare the performance of the O-CNN with other existing 3D CNN solutions and demonstrate the efficiency and efficacy of O-CNN in three shape analysis tasks, including object classification, shape retrieval, and shape segmentation.

Cited by in corpus (53)

• Parallel convolution processing using an integrated photonic tensor core
• MeshCNN: A Network with an Edge
• Linking Points With Labels in 3D: A Review of Point Cloud Semantic Segmentation
• Image-based 3D Object Reconstruction: State-of-the-Art and Trends in the Deep Learning Era
• Pix2Vox: Context-aware 3D Reconstruction from Single and Multi-view Images
• Deep Learning-based 3D Point Cloud Classification: A Systematic Survey and Outlook
• Monte Carlo Convolution for Learning on Non-Uniformly Sampled Point Clouds
• OctAttention: Octree-Based Large-Scale Contexts Model for Point Cloud Compression
• Enable Deep Learning on Mobile Devices: Methods, Systems, and Applications
• Subdivision-Based Mesh Convolution Networks
• Locally Attentional SDF Diffusion for Controllable 3D Shape Generation
• OctFormer: Octree-based Transformers for 3D Point Clouds
• Adaptive O-CNN: A Patch-based Deep Representation of 3D Shapes
• Dual Octree Graph Networks for Learning Adaptive Volumetric Shape Representations
• Hierarchical Attention Learning of Scene Flow in 3D Point Clouds
• DNF-Net: a Deep Normal Filtering Network for Mesh Denoising
• LassoNet: Deep Lasso-Selection of 3D Point Clouds
• Two-Stream Graph Convolutional Network for Intra-oral Scanner Image Segmentation
• GCN-Denoiser: Mesh Denoising with Graph Convolutional Networks
• Locomotion Policy Guided Traversability Learning using Volumetric Representations of Complex Environments
• Laplacian2Mesh: Laplacian-Based Mesh Understanding
• Advancements in Point Cloud-Based 3D Defect Detection and Classification for Industrial Systems: A Comprehensive Survey
• Deep Part Induction from Articulated Object Pairs
• DeepWarp: DNN-based Nonlinear Deformation
• Point2SpatialCapsule: Aggregating Features and Spatial Relationships of Local Regions on Point Clouds using Spatial-aware Capsules
• Fault Diagnosis in Microelectronics Attachment via Deep Learning Analysis of 3D Laser Scans
• Learning Part Boundaries from 3D Point Clouds
• 3D Mesh Segmentation via Multi-branch 1D Convolutional Neural Networks
• H-CNN: Spatial Hashing Based CNN for 3D Shape Analysis
• Spatial Transformer for 3D Point Clouds
• PIG-Net: Inception based Deep Learning Architecture for 3D Point Cloud Segmentation
• Point Set Voting for Partial Point Cloud Analysis
• PSNet: Fast Data Structuring for Hierarchical Deep Learning on Point Cloud
• 3D Instance Segmentation of MVS Buildings
• fVDB: A Deep-Learning Framework for Sparse, Large-Scale, and High-Performance Spatial Intelligence
• Learning to Grasp on the Moon from 3D Octree Observations with Deep Reinforcement Learning
• PointSCNet: Point Cloud Structure and Correlation Learning Based on Space Filling Curve-Guided Sampling
• Unsupervised deep learning for semantic segmentation of multispectral LiDAR forest point clouds
• Multi-feature super-resolution network for cloth wrinkle synthesis
• Learning Mesh Representations via Binary Space Partitioning Tree Networks
• Bayesian regional moment tensor from ocean bottom seismograms recorded in the Lesser Antilles: Implications for regional stress field
• Learning Material-Aware Local Descriptors for 3D Shapes
• GeoFormer: Learning Point Cloud Completion with Tri-Plane Integrated Transformer
• Projection-based Point Convolution for Efficient Point Cloud Segmentation
• Semantic Segmentation of Urban Textured Meshes Through Point Sampling
• ePointDA: An End-to-End Simulation-to-Real Domain Adaptation Framework for LiDAR Point Cloud Segmentation
• Bringing Attention to CAD: Boundary Representation Learning via Transformer
• PointSplit: Towards On-device 3D Object Detection with Heterogeneous Low-power Accelerators
• Surface Geometry Processing: An Efficient Normal-based Detail Representation
• Scan2Part: Fine-grained and Hierarchical Part-level Understanding of Real-World 3D Scans
• SS4D: Native 4D Generative Model via Structured Spacetime Latents
• Nexus: Native Mesh Generation with Diffusion
• A Survey on Deep Learning Architectures for Point Cloud Classification and Segmentation