ComplementMe: Weakly-Supervised Component Suggestions for 3D Modeling

Abstract

Assembly-based tools provide a powerful modeling paradigm for non-expert shape designers. However, choosing a component from a large shape repository and aligning it to a partial assembly can become a daunting task. In this paper we describe novel neural network architectures for suggesting complementary components and their placement for an incomplete 3D part assembly. Unlike most existing techniques, our networks are trained on unlabeled data obtained from public online repositories, and do not rely on consistent part segmentations or labels. Absence of labels poses a challenge in indexing the database of parts for the retrieval. We address it by jointly training embedding and retrieval networks, where the first indexes parts by mapping them to a low-dimensional feature space, and the second maps partial assemblies to appropriate complements. The combinatorial nature of part arrangements poses another challenge, since the retrieval network is not a function: several complements can be appropriate for the same input. Thus, instead of predicting a single output, we train our network to predict a probability distribution over the space of part embeddings. This allows our method to deal with ambiguities and naturally enables a UI that seamlessly integrates user preferences into the design process. We demonstrate that our method can be used to design complex shapes with minimal or no user input. To evaluate our approach we develop a novel benchmark for component suggestion systems demonstrating significant improvement over state-of-the-art techniques.

SIGGRAPH Asia 2017. 12 pages

References in corpus (6)

• Adam: A Method for Stochastic Optimization
• Conditional Generative Adversarial Nets
• ShapeNet: An Information-Rich 3D Model Repository
• Learning a Probabilistic Latent Space of Object Shapes via 3D Generative-Adversarial Modeling
• Semi-Supervised Learning with Deep Generative Models
• Learning Hierarchical Shape Segmentation and Labeling from Online Repositories

Cited by in corpus (6)

• P2P-NET: Bidirectional Point Displacement Net for Shape Transform
• ShapeAssembly: Learning to Generate Programs for 3D Shape Structure Synthesis
• 3D Part Assembly Generation with Instance Encoded Transformer
• Learning to Group and Label Fine-Grained Shape Components
• Predictive and Generative Neural Networks for Object Functionality
• Roominoes: Generating Novel 3D Floor Plans From Existing 3D Rooms