PQ-NET: A Generative Part Seq2Seq Network for 3D Shapes

Abstract

We introduce PQ-NET, a deep neural network which represents and generates 3D shapes via sequential part assembly. The input to our network is a 3D shape segmented into parts, where each part is first encoded into a feature representation using a part autoencoder. The core component of PQ-NET is a sequence-to-sequence or Seq2Seq autoencoder which encodes a sequence of part features into a latent vector of fixed size, and the decoder reconstructs the 3D shape, one part at a time, resulting in a sequential assembly. The latent space formed by the Seq2Seq encoder encodes both part structure and fine part geometry. The decoder can be adapted to perform several generative tasks including shape autoencoding, interpolation, novel shape generation, and single-view 3D reconstruction, where the generated shapes are all composed of meaningful parts.

arXiv commentAccepted to CVPR 2020. Code available at https://github.com/ChrisWu1997/PQ-NET

References in corpus (8)

• Sequence to Sequence Learning with Neural Networks
• Learning a Probabilistic Latent Space of Object Shapes via 3D Generative-Adversarial Modeling
• Deep Convolutional Inverse Graphics Network
• Matryoshka Networks: Predicting 3D Geometry via Nested Shape Layers
• PointFlow: 3D Point Cloud Generation with Continuous Normalizing Flows
• Learning Part Generation and Assembly for Structure-aware Shape Synthesis
• Im2Struct: Recovering 3D Shape Structure from a Single RGB Image
• Probabilistic Reconstruction Networks for 3D Shape Inference from a Single Image

Cited by in corpus (5)

• PT2PC: Learning to Generate 3D Point Cloud Shapes from Part Tree Conditions
• Multimodal Shape Completion via Conditional Generative Adversarial Networks
• A Survey on Deep Geometry Learning: From a Representation Perspective
• Learning 3D Part Assembly from a Single Image
• RISA-Net: Rotation-Invariant Structure-Aware Network for Fine-Grained 3D Shape Retrieval