Sketch2Model: View-Aware 3D Modeling from Single Free-Hand Sketches

Abstract

We investigate the problem of generating 3D meshes from single free-hand sketches, aiming at fast 3D modeling for novice users. It can be regarded as a single-view reconstruction problem, but with unique challenges, brought by the variation and conciseness of sketches. Ambiguities in poorly-drawn sketches could make it hard to determine how the sketched object is posed. In this paper, we address the importance of viewpoint specification for overcoming such ambiguities, and propose a novel view-aware generation approach. By explicitly conditioning the generation process on a given viewpoint, our method can generate plausible shapes automatically with predicted viewpoints, or with specified viewpoints to help users better express their intentions. Extensive evaluations on various datasets demonstrate the effectiveness of our view-aware design in solving sketch ambiguities and improving reconstruction quality.

arXiv commentAccepted to CVPR 2021

References in corpus (6)

• Perspective Transformer Nets: Learning Single-View 3D Object Reconstruction without 3D Supervision
• 3D Sketching using Multi-View Deep Volumetric Prediction
• Soft Rasterizer: A Differentiable Renderer for Image-based 3D Reasoning
• Sketch-a-Net that Beats Humans
• Self-supervised Single-view 3D Reconstruction via Semantic Consistency
• 3D Shape Reconstruction from Free-Hand Sketches