Combinatorial 3D Shape Generation via Sequential Assembly

Abstract

Sequential assembly with geometric primitives has drawn attention in robotics and 3D vision since it yields a practical blueprint to construct a target shape. However, due to its combinatorial property, a greedy method falls short of generating a sequence of volumetric primitives. To alleviate this consequence induced by a huge number of feasible combinations, we propose a combinatorial 3D shape generation framework. The proposed framework reflects an important aspect of human generation processes in real life -- we often create a 3D shape by sequentially assembling unit primitives with geometric constraints. To find the desired combination regarding combination evaluations, we adopt Bayesian optimization, which is able to exploit and explore efficiently the feasible regions constrained by the current primitive placements. An evaluation function conveys global structure guidance for an assembly process and stability in terms of gravity and external forces simultaneously. Experimental results demonstrate that our method successfully generates combinatorial 3D shapes and simulates more realistic generation processes. We also introduce a new dataset for combinatorial 3D shape generation. All the codes are available at \url{https://github.com/POSTECH-CVLab/Combinatorial-3D-Shape-Generation}.

arXiv comment14 pages, 20 figures, 1 table, presented at NeurIPS 2020 Workshop on Machine Learning for Engineering Modeling, Simulation, and Design

References in corpus (10)

• Auto-Encoding Variational Bayes
• Practical Bayesian Optimization of Machine Learning Algorithms
• A Tutorial on Bayesian Optimization of Expensive Cost Functions, with Application to Active User Modeling and Hierarchical Reinforcement Learning
• Learning a Probabilistic Latent Space of Object Shapes via 3D Generative-Adversarial Modeling
• Open3D: A Modern Library for 3D Data Processing
• Learning Representations and Generative Models for 3D Point Clouds
• A Tutorial on Bayesian Optimization
• Bayesian Optimization of Combinatorial Structures
• PolyGen: An Autoregressive Generative Model of 3D Meshes
• Tight Regret Bounds for Bayesian Optimization in One Dimension

Cited by in corpus (2)

• Brick-by-Brick: Combinatorial Construction with Deep Reinforcement Learning
• Building LEGO Using Deep Generative Models of Graphs