Render for CNN: Viewpoint Estimation in Images Using CNNs Trained with Rendered 3D Model Views
arXiv:1505.05641
Abstract
Object viewpoint estimation from 2D images is an essential task in computer vision. However, two issues hinder its progress: scarcity of training data with viewpoint annotations, and a lack of powerful features. Inspired by the growing availability of 3D models, we propose a framework to address both issues by combining render-based image synthesis and CNNs. We believe that 3D models have the potential in generating a large number of images of high variation, which can be well exploited by deep CNN with a high learning capacity. Towards this goal, we propose a scalable and overfit-resistant image synthesis pipeline, together with a novel CNN specifically tailored for the viewpoint estimation task. Experimentally, we show that the viewpoint estimation from our pipeline can significantly outperform state-of-the-art methods on PASCAL 3D+ benchmark.
References in corpus (5)
Cited by in corpus (57)
- Learning from Synthetic Humans
- Domain Adaptation for Visual Applications: A Comprehensive Survey
- Domain Randomization for Transferring Deep Neural Networks from Simulation to the Real World
- MoCap-guided Data Augmentation for 3D Pose Estimation in the Wild
- PoseCNN: A Convolutional Neural Network for 6D Object Pose Estimation in Cluttered Scenes
- Discovery of Latent 3D Keypoints via End-to-end Geometric Reasoning
- Deep Domain Adaptation Based Video Smoke Detection using Synthetic Smoke Images
- Synthesizing Training Images for Boosting Human 3D Pose Estimation
- Pix3D: Dataset and Methods for Single-Image 3D Shape Modeling
- Object Detection Using Deep CNNs Trained on Synthetic Images
- Rethinking Reprojection: Closing the Loop for Pose-aware ShapeReconstruction from a Single Image
- Transformation-Grounded Image Generation Network for Novel 3D View Synthesis
- SurfNet: Generating 3D shape surfaces using deep residual networks
- 3D Bounding Box Estimation Using Deep Learning and Geometry
- 3D Shape Induction from 2D Views of Multiple Objects
- The challenge of simultaneous object detection and pose estimation: a comparative study
- Deep Cuboid Detection: Beyond 2D Bounding Boxes
- Crafting a multi-task CNN for viewpoint estimation
- FacePoseNet: Making a Case for Landmark-Free Face Alignment
- Real-Time Seamless Single Shot 6D Object Pose Prediction
- Synthetic to Real Adaptation with Generative Correlation Alignment Networks
- Automatic Generation of Constrained Furniture Layouts
- ApolloCar3D: A Large 3D Car Instance Understanding Benchmark for Autonomous Driving
- Novel Views of Objects from a Single Image
- Pixels, voxels, and views: A study of shape representations for single view 3D object shape prediction
- DeepContext: Context-Encoding Neural Pathways for 3D Holistic Scene Understanding
- Deep Supervision with Shape Concepts for Occlusion-Aware 3D Object Parsing
- Visual Compiler: Synthesizing a Scene-Specific Pedestrian Detector and Pose Estimator
- 3D-RETR: End-to-End Single and Multi-View 3D Reconstruction with Transformers
- Domain Adaptation from Synthesis to Reality in Single-model Detector for Video Smoke Detection
- Frankenstein: Learning Deep Face Representations using Small Data
- 3D Reconstruction of Simple Objects from A Single View Silhouette Image
- Learning Compositional Visual Concepts with Mutual Consistency
- Generative Model with Coordinate Metric Learning for Object Recognition Based on 3D Models
- SampleAhead: Online Classifier-Sampler Communication for Learning from Synthesized Data
- Shape Generation using Spatially Partitioned Point Clouds
- Pano2CAD: Room Layout From A Single Panorama Image
- RealPoint3D: Point Cloud Generation from a Single Image with Complex Background
- Physically-Based Rendering for Indoor Scene Understanding Using Convolutional Neural Networks
- A spatiotemporal model with visual attention for video classification
- Improved Deep Learning of Object Category using Pose Information
- Real-time Deep Pose Estimation with Geodesic Loss for Image-to-Template Rigid Registration
- A Mixed Classification-Regression Framework for 3D Pose Estimation from 2D Images
- Single-View 3D Object Reconstruction from Shape Priors in Memory
- Click Here: Human-Localized Keypoints as Guidance for Viewpoint Estimation
- From Images to 3D Shape Attributes
- Object-Centric Photometric Bundle Adjustment with Deep Shape Prior
- Synthesizing 3D Shapes from Silhouette Image Collections using Multi-projection Generative Adversarial Networks
- Learning to Recognize Objects by Retaining other Factors of Variation
- Fast 3D Pose Refinement with RGB Images
- REST: Real-to-Synthetic Transform for Illumination Invariant Camera Localization
- iSPA-Net: Iterative Semantic Pose Alignment Network
- A Dataset for Developing and Benchmarking Active Vision
- Shape from Shading through Shape Evolution
- A Variational Observation Model of 3D Object for Probabilistic Semantic SLAM
- From Virtual to Real World Visual Perception using Domain Adaptation -- The DPM as Example
- Context-based Object Viewpoint Estimation: A 2D Relational Approach