PhotoShape: Photorealistic Materials for Large-Scale Shape Collections
arXiv:1809.09761 · doi:10.1145/3272127.3275066
Abstract
Existing online 3D shape repositories contain thousands of 3D models but lack photorealistic appearance. We present an approach to automatically assign high-quality, realistic appearance models to large scale 3D shape collections. The key idea is to jointly leverage three types of online data -- shape collections, material collections, and photo collections, using the photos as reference to guide assignment of materials to shapes. By generating a large number of synthetic renderings, we train a convolutional neural network to classify materials in real photos, and employ 3D-2D alignment techniques to transfer materials to different parts of each shape model. Our system produces photorealistic, relightable, 3D shapes (PhotoShapes).
arXiv commentTo be presented at SIGGRAPH Asia 2018. Project page: https://keunhong.com/publications/photoshape/
References in corpus (7)
- Efficient Inference in Fully Connected CRFs with Gaussian Edge Potentials
- ShapeNet: An Information-Rich 3D Model Repository
- Texture Synthesis Using Convolutional Neural Networks
- Modeling Surface Appearance from a Single Photograph using Self-augmented Convolutional Neural Networks
- Playing for Data: Ground Truth from Computer Games
- A Lightweight Approach for On-the-Fly Reflectance Estimation
- Material Editing Using a Physically Based Rendering Network
Cited by in corpus (10)
- GRAF: Generative Radiance Fields for 3D-Aware Image Synthesis
- BlockGAN: Learning 3D Object-aware Scene Representations from Unlabelled Images
- EASI-Tex: Edge-Aware Mesh Texturing from Single Image
- OpenRooms: An End-to-End Open Framework for Photorealistic Indoor Scene Datasets
- L3DG: Latent 3D Gaussian Diffusion
- Editing Conditional Radiance Fields
- Review: deep learning on 3D point clouds
- MatCLIP: Light- and Shape-Insensitive Assignment of PBR Material Models
- Advances in Neural 3D Mesh Texturing: A Survey
- GARDEN: Gravity-Aligned Reconstruction of Disentangled ENvironments from RGB images