Photo-Geometric Autoencoding to Learn 3D Objects from Unlabelled Images
arXiv:1906.01568
Abstract
We show that generative models can be used to capture visual geometry constraints statistically. We use this fact to infer the 3D shape of object categories from raw single-view images. Differently from prior work, we use no external supervision, nor do we use multiple views or videos of the objects. We achieve this by a simple reconstruction task, exploiting the symmetry of the objects' shape and albedo. Specifically, given a single image of the object seen from an arbitrary viewpoint, our model predicts a symmetric canonical view, the corresponding 3D shape and a viewpoint transformation, and trains with the goal of reconstructing the input view, resembling an auto-encoder. Our experiments show that this method can recover the 3D shape of human faces, cat faces, and cars from single view images, without supervision. On benchmarks, we demonstrate superior accuracy compared to other methods that use supervision at the level of 2D image correspondences.
Appendix included, 17 pages. Project page: https://elliottwu.com/projects/unsup3d
References in corpus (8)
- Depth Map Prediction from a Single Image using a Multi-Scale Deep Network
- DeMoN: Depth and Motion Network for Learning Monocular Stereo
- Discovery of Latent 3D Keypoints via End-to-end Geometric Reasoning
- Deforming Autoencoders: Unsupervised Disentangling of Shape and Appearance
- Visual Object Networks: Image Generation with Disentangled 3D Representation
- Unsupervised Adversarial Learning of 3D Human Pose from 2D Joint Locations
- Unsupervised Depth Estimation, 3D Face Rotation and Replacement
- Unsupervised 3D Shape Learning from Image Collections in the Wild