MeronymNet: A Hierarchical Approach for Unified and Controllable Multi-Category Object Generation

Abstract

We introduce MeronymNet, a novel hierarchical approach for controllable, part-based generation of multi-category objects using a single unified model. We adopt a guided coarse-to-fine strategy involving semantically conditioned generation of bounding box layouts, pixel-level part layouts and ultimately, the object depictions themselves. We use Graph Convolutional Networks, Deep Recurrent Networks along with custom-designed Conditional Variational Autoencoders to enable flexible, diverse and category-aware generation of 2-D objects in a controlled manner. The performance scores for generated objects reflect MeronymNet's superior performance compared to multiple strong baselines and ablative variants. We also showcase MeronymNet's suitability for controllable object generation and interactive object editing at various levels of structural and semantic granularity.

arXiv commentAccepted at ACM Multimedia (ACMMM) 2021 [ORAL] . Website : https://meronymnet.github.io/. arXiv admin note: text overlap with arXiv:2006.00190

References in corpus (18)

• Adam: A Method for Stochastic Optimization
• Auto-Encoding Variational Bayes
• Semi-Supervised Classification with Graph Convolutional Networks
• GANs Trained by a Two Time-Scale Update Rule Converge to a Local Nash Equilibrium
• Large Scale GAN Training for High Fidelity Natural Image Synthesis
• UnitBox: An Advanced Object Detection Network
• Tutorial on Variational Autoencoders
• Variational Graph Auto-Encoders
• Semantic Graph Convolutional Networks for 3D Human Pose Regression
• Constrained Graph Variational Autoencoders for Molecule Design
• SAGNet:Structure-aware Generative Network for 3D-Shape Modeling
• MC-GAN: Multi-conditional Generative Adversarial Network for Image Synthesis
• LayoutGAN: Generating Graphic Layouts with Wireframe Discriminators
• Scene Graph Generation with External Knowledge and Image Reconstruction
• Semantics Disentangling for Text-to-Image Generation
• Semantic Bottleneck Scene Generation
• Image Synthesis From Reconfigurable Layout and Style
• Controllable Image Synthesis via SegVAE