Introducing Pose Consistency and Warp-Alignment for Self-Supervised 6D Object Pose Estimation in Color Images

Abstract

Most successful approaches to estimate the 6D pose of an object typically train a neural network by supervising the learning with annotated poses in real world images. These annotations are generally expensive to obtain and a common workaround is to generate and train on synthetic scenes, with the drawback of limited generalisation when the model is deployed in the real world. In this work, a two-stage 6D object pose estimator framework that can be applied on top of existing neural-network-based approaches and that does not require pose annotations on real images is proposed. The first self-supervised stage enforces the pose consistency between rendered predictions and real input images, narrowing the gap between the two domains. The second stage fine-tunes the previously trained model by enforcing the photometric consistency between pairs of different object views, where one image is warped and aligned to match the view of the other and thus enabling their comparison. In the absence of both real image annotations and depth information, applying the proposed framework on top of two recent approaches results in state-of-the-art performance when compared to methods trained only on synthetic data, domain adaptation baselines and a concurrent self-supervised approach on LINEMOD, LINEMOD OCCLUSION and HomebrewedDB datasets.

arXiv commentAccepted to 3DV'2020 as Oral

References in corpus (24)

• Adam: A Method for Stochastic Optimization
• Very Deep Convolutional Networks for Large-Scale Image Recognition
• Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks
• Distilling the Knowledge in a Neural Network
• Deep Domain Confusion: Maximizing for Domain Invariance
• Deep Learning with Limited Numerical Precision
• Binarized Neural Networks
• The Unreasonable Effectiveness of Deep Features as a Perceptual Metric
• A Learned Representation For Artistic Style
• Learning multiple visual domains with residual adapters
• Beyond Sharing Weights for Deep Domain Adaptation
• Learning Structured Sparsity in Deep Neural Networks
• Model compression via distillation and quantization
• Multi-Task Deep Neural Networks for Natural Language Understanding
• Domain Randomization for Transferring Deep Neural Networks from Simulation to the Real World
• DSD: Dense-Sparse-Dense Training for Deep Neural Networks
• Accelerating 3D Deep Learning with PyTorch3D
• Universal representations:The missing link between faces, text, planktons, and cat breeds
• Pix3D: Dataset and Methods for Single-Image 3D Shape Modeling
• BigHand2.2M Benchmark: Hand Pose Dataset and State of the Art Analysis
• Drop to Adapt: Learning Discriminative Features for Unsupervised Domain Adaptation
• Multi-Task Deep Networks for Depth-Based 6D Object Pose and Joint Registration in Crowd Scenarios
• An Improved Self-supervised GAN via Adversarial Training
• Self-supervised 3D Shape and Viewpoint Estimation from Single Images for Robotics

Cited by in corpus (1)

• ACR-Pose: Adversarial Canonical Representation Reconstruction Network for Category Level 6D Object Pose Estimation