MeshCNN: A Network with an Edge

Abstract

Polygonal meshes provide an efficient representation for 3D shapes. They explicitly capture both shape surface and topology, and leverage non-uniformity to represent large flat regions as well as sharp, intricate features. This non-uniformity and irregularity, however, inhibits mesh analysis efforts using neural networks that combine convolution and pooling operations. In this paper, we utilize the unique properties of the mesh for a direct analysis of 3D shapes using MeshCNN, a convolutional neural network designed specifically for triangular meshes. Analogous to classic CNNs, MeshCNN combines specialized convolution and pooling layers that operate on the mesh edges, by leveraging their intrinsic geodesic connections. Convolutions are applied on edges and the four edges of their incident triangles, and pooling is applied via an edge collapse operation that retains surface topology, thereby, generating new mesh connectivity for the subsequent convolutions. MeshCNN learns which edges to collapse, thus forming a task-driven process where the network exposes and expands the important features while discarding the redundant ones. We demonstrate the effectiveness of our task-driven pooling on various learning tasks applied to 3D meshes.

For a two-minute explanation video see https://bit.ly/meshcnnvideo

References in corpus (21)

• Very Deep Convolutional Networks for Large-Scale Image Recognition
• Geometric deep learning: going beyond Euclidean data
• Spectral Networks and Locally Connected Networks on Graphs
• Convolutional Neural Networks on Graphs with Fast Localized Spectral Filtering
• Deep Convolutional Networks on Graph-Structured Data
• OverFeat: Integrated Recognition, Localization and Detection using Convolutional Networks
• O-CNN: Octree-based Convolutional Neural Networks for 3D Shape Analysis
• Hierarchical Graph Representation Learning with Differentiable Pooling
• Learning Convolutional Neural Networks for Graphs
• MeshCNN: A Network with an Edge
• Diffusion-Convolutional Neural Networks
• Generative and Discriminative Voxel Modeling with Convolutional Neural Networks
• PCPNET: Learning Local Shape Properties from Raw Point Clouds
• FPNN: Field Probing Neural Networks for 3D Data
• Towards Sparse Hierarchical Graph Classifiers
• Learning shape correspondence with anisotropic convolutional neural networks
• Robust Spatial Filtering with Graph Convolutional Neural Networks
• Spherical CNNs on Unstructured Grids
• Dual-Primal Graph Convolutional Networks
• Generating 3D faces using Convolutional Mesh Autoencoders
• Comparison of parallel sorting algorithms

Cited by in corpus (96)

• MeshCNN: A Network with an Edge
• Point2Mesh: A Self-Prior for Deformable Meshes
• Skeleton-Aware Networks for Deep Motion Retargeting
• Subdivision-Based Mesh Convolution Networks
• Learning Skeletal Articulations with Neural Blend Shapes
• FEA-Net: A Physics-guided Data-driven Model for Efficient Mechanical Response Prediction
• SegGroup: Seg-Level Supervision for 3D Instance and Semantic Segmentation
• Hierarchical Attention Learning of Scene Flow in 3D Point Clouds
• DNF-Net: a Deep Normal Filtering Network for Mesh Denoising
• Kaolin: A PyTorch Library for Accelerating 3D Deep Learning Research
• Orienting Point Clouds with Dipole Propagation
• GCN-Denoiser: Mesh Denoising with Graph Convolutional Networks
• Laplacian2Mesh: Laplacian-Based Mesh Understanding
• CNNs on Surfaces using Rotation-Equivariant Features
• PRS-Net: Planar Reflective Symmetry Detection Net for 3D Models
• Deep Geometric Texture Synthesis
• Multi-Scale Progressive Fusion Network for Single Image Deraining
• Towards Reusable Surrogate Models: Graph-Based Transfer Learning on Trusses
• Learning Mesh-Based Simulation with Graph Networks
• Primal-Dual Mesh Convolutional Neural Networks
• Fully Convolutional Mesh Autoencoder using Efficient Spatially Varying Kernels
• Deep Learning Based 3D Segmentation: A Survey
• DeltaConv: Anisotropic Operators for Geometric Deep Learning on Point Clouds
• Style2Fab: Functionality-Aware Segmentation for Fabricating Personalized 3D Models with Generative AI
• Mesh Neural Networks for SE(3)-Equivariant Hemodynamics Estimation on the Artery Wall
• Deep Learning for Real-Time Aerodynamic Evaluations of Arbitrary Vehicle Shapes
• MGCN: Descriptor Learning using Multiscale GCNs
• SDM-NET: Deep Generative Network for Structured Deformable Mesh
• Adversarial Generation of Continuous Implicit Shape Representations
• Applying Convolutional Neural Networks to Data on Unstructured Meshes with Space-Filling Curves
• ZerNet: Convolutional Neural Networks on Arbitrary Surfaces via Zernike Local Tangent Space Estimation
• Complete & Label: A Domain Adaptation Approach to Semantic Segmentation of LiDAR Point Clouds
• PDE-GCN: Novel Architectures for Graph Neural Networks Motivated by Partial Differential Equations
• Rigid Motion Invariant Statistical Shape Modeling based on Discrete Fundamental Forms
• 3D-FUTURE: 3D Furniture shape with TextURE
• Juggling With Representations: On the Information Transfer Between Imagery, Point Clouds, and Meshes for Multi-Modal Semantics
• RigNet: Neural Rigging for Articulated Characters
• Structure-Aware Completion of Photogrammetric Meshes in Urban Road Environment
• Learning Mesh Representations via Binary Space Partitioning Tree Networks
• BSP-Net: Generating Compact Meshes via Binary Space Partitioning
• Learning Acoustic Scattering Fields for Dynamic Interactive Sound Propagation
• Higher-Order Function Networks for Learning Composable 3D Object Representations
• The Whole Is Greater Than the Sum of Its Nonrigid Parts
• Combining Differentiable PDE Solvers and Graph Neural Networks for Fluid Flow Prediction
• SelfVoxeLO: Self-supervised LiDAR Odometry with Voxel-based Deep Neural Networks
• Multimodal Semi-Supervised Learning for 3D Objects
• DiffGCN: Graph Convolutional Networks via Differential Operators and Algebraic Multigrid Pooling
• Coordinate Independent Convolutional Networks -- Isometry and Gauge Equivariant Convolutions on Riemannian Manifolds
• AutoMate: A Dataset and Learning Approach for Automatic Mating of CAD Assemblies
• HeterSkinNet: A Heterogeneous Network for Skin Weights Prediction
• Learning Gradient Fields for Shape Generation
• Learning the Geodesic Embedding with Graph Neural Networks
• PvDeConv: Point-Voxel Deconvolution for Autoencoding CAD Construction in 3D
• MeshWalker: Deep Mesh Understanding by Random Walks
• Deformation Recovery: Localized Learning for Detail-Preserving Deformations
• Deep Mesh Prior: Unsupervised Mesh Restoration using Graph Convolutional Networks
• IntrA: 3D Intracranial Aneurysm Dataset for Deep Learning
• Bringing Attention to CAD: Boundary Representation Learning via Transformer
• Continuous Geodesic Convolutions for Learning on 3D Shapes
• TilinGNN: Learning to Tile with Self-Supervised Graph Neural Network
• Unsupervised 3D Learning for Shape Analysis via Multiresolution Instance Discrimination
• Neural ShDF: Reviving an Efficient and Consistent Mesh Segmentation Method
• iSeg: Interactive 3D Segmentation via Interactive Attention
• NeSF: Neural Semantic Fields for Generalizable Semantic Segmentation of 3D Scenes
• Voint Cloud: Multi-View Point Cloud Representation for 3D Understanding
• Learning Self-Prior for Mesh Inpainting Using Self-Supervised Graph Convolutional Networks
• Neural Mesh Flow: 3D Manifold Mesh Generation via Diffeomorphic Flows
• QuadConv: Quadrature-Based Convolutions with Applications to Non-Uniform PDE Data Compression
• HodgeNet: Learning Spectral Geometry on Triangle Meshes
• A Survey on Deep Geometry Learning: From a Representation Perspective
• StructEdit: Learning Structural Shape Variations
• Learning Feature Aggregation for Deep 3D Morphable Models
• Scaling Up Bayesian Uncertainty Quantification for Inverse Problems using Deep Neural Networks
• Random Walks for Adversarial Meshes
• Self-Sampling for Neural Point Cloud Consolidation
• A Two-step Surface-based 3D Deep Learning Pipeline for Segmentation of Intracranial Aneurysms
• Cross-modal Center Loss
• LCollision: Fast Generation of Collision-Free Human Poses using Learned Non-Penetration Constraints
• CageNet: A Meta-Framework for Learning on Wild Meshes
• RISA-Net: Rotation-Invariant Structure-Aware Network for Fine-Grained 3D Shape Retrieval
• PoissonNet: A Local-Global Approach for Learning on Surfaces
• Neural Trees for Learning on Graphs
• Dense Graph Convolutional Neural Networks on 3D Meshes for 3D Object Segmentation and Classification
• 3D Shapes Local Geometry Codes Learning with SDF
• Deep Hybrid Self-Prior for Full 3D Mesh Generation
• Deep 3D Mesh Watermarking with Self-Adaptive Robustness
• EuLearn: A 3D database for learning Euler characteristics
• Depth-Enhanced Feature Pyramid Network for Occlusion-Aware Verification of Buildings from Oblique Images
• Towards Part-Based Understanding of RGB-D Scans
• Picasso: A CUDA-based Library for Deep Learning over 3D Meshes
• Implementing a Machine Learning Deformer for CG Crowds: Our Journey
• Deep Patch-based Human Segmentation
• Rotation Equivariant 3D Hand Mesh Generation from a Single RGB Image
• Point-based Acoustic Scattering for Interactive Sound Propagation via Surface Encoding
• A 3D mesh convolution-based autoencoder for geometry compression
• Training or Architecture? How to Incorporate Invariance in Neural Networks