Scan2Part: Fine-grained and Hierarchical Part-level Understanding of Real-World 3D Scans
arXiv:2206.02366 · doi:10.5220/0010848200003124
Abstract
We propose Scan2Part, a method to segment individual parts of objects in real-world, noisy indoor RGB-D scans. To this end, we vary the part hierarchies of objects in indoor scenes and explore their effect on scene understanding models. Specifically, we use a sparse U-Net-based architecture that captures the fine-scale detail of the underlying 3D scan geometry by leveraging a multi-scale feature hierarchy. In order to train our method, we introduce the Scan2Part dataset, which is the first large-scale collection providing detailed semantic labels at the part level in the real-world setting. In total, we provide 242,081 correspondences between 53,618 PartNet parts of 2,477 ShapeNet objects and 1,506 ScanNet scenes, at two spatial resolutions of 2 cm and 5 cm. As output, we are able to predict fine-grained per-object part labels, even when the geometry is coarse or partially missing.
arXiv commentIn Proceedings of the 17th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications
References in corpus (13)
- Adam: A Method for Stochastic Optimization
- ShapeNet: An Information-Rich 3D Model Repository
- O-CNN: Octree-based Convolutional Neural Networks for 3D Shape Analysis
- Joint 2D-3D-Semantic Data for Indoor Scene Understanding
- Semantic Instance Segmentation with a Discriminative Loss Function
- The Replica Dataset: A Digital Replica of Indoor Spaces
- Learning Object Bounding Boxes for 3D Instance Segmentation on Point Clouds
- Superhuman Accuracy on the SNEMI3D Connectomics Challenge
- MASC: Multi-scale Affinity with Sparse Convolution for 3D Instance Segmentation
- SAGNet:Structure-aware Generative Network for 3D-Shape Modeling
- StructureNet: Hierarchical Graph Networks for 3D Shape Generation
- 3D Graph Embedding Learning with a Structure-aware Loss Function for Point Cloud Semantic Instance Segmentation
- Point Cloud Instance Segmentation using Probabilistic Embeddings