3D Object Detection and Pose Estimation of Unseen Objects in Color Images with Local Surface Embeddings

Abstract

We present an approach for detecting and estimating the 3D poses of objects in images that requires only an untextured CAD model and no training phase for new objects. Our approach combines Deep Learning and 3D geometry: It relies on an embedding of local 3D geometry to match the CAD models to the input images. For points at the surface of objects, this embedding can be computed directly from the CAD model; for image locations, we learn to predict it from the image itself. This establishes correspondences between 3D points on the CAD model and 2D locations of the input images. However, many of these correspondences are ambiguous as many points may have similar local geometries. We show that we can use Mask-RCNN in a class-agnostic way to detect the new objects without retraining and thus drastically limit the number of possible correspondences. We can then robustly estimate a 3D pose from these discriminative correspondences using a RANSAC- like algorithm. We demonstrate the performance of this approach on the T-LESS dataset, by using a small number of objects to learn the embedding and testing it on the other objects. Our experiments show that our method is on par or better than previous methods.

References in corpus (10)

• Learning Transferable Features with Deep Adaptation Networks
• Domain Separation Networks
• Pix2Pose: Pixel-Wise Coordinate Regression of Objects for 6D Pose Estimation
• Learning Descriptors for Object Recognition and 3D Pose Estimation
• DeepIM: Deep Iterative Matching for 6D Pose Estimation
• DRIT++: Diverse Image-to-Image Translation via Disentangled Representations
• 3D Object Instance Recognition and Pose Estimation Using Triplet Loss with Dynamic Margin
• PVNet: Pixel-wise Voting Network for 6DoF Pose Estimation
• When Regression Meets Manifold Learning for Object Recognition and Pose Estimation
• 3D Pose Estimation for Fine-Grained Object Categories

Cited by in corpus (5)

• Deep Learning-Based Object Pose Estimation: A Comprehensive Survey
• DISP6D: Disentangled Implicit Shape and Pose Learning for Scalable 6D Pose Estimation
• PoseContrast: Class-Agnostic Object Viewpoint Estimation in the Wild with Pose-Aware Contrastive Learning
• LocaliseBot: Multi-view 3D object localisation with differentiable rendering for robot grasping
• Towards Symmetry-sensitive Pose Estimation: A Rotation Representation for Symmetric Object Classes