DPODv2: Dense Correspondence-Based 6 DoF Pose Estimation

Abstract

We propose a three-stage 6 DoF object detection method called DPODv2 (Dense Pose Object Detector) that relies on dense correspondences. We combine a 2D object detector with a dense correspondence estimation network and a multi-view pose refinement method to estimate a full 6 DoF pose. Unlike other deep learning methods that are typically restricted to monocular RGB images, we propose a unified deep learning network allowing different imaging modalities to be used (RGB or Depth). Moreover, we propose a novel pose refinement method, that is based on differentiable rendering. The main concept is to compare predicted and rendered correspondences in multiple views to obtain a pose which is consistent with predicted correspondences in all views. Our proposed method is evaluated rigorously on different data modalities and types of training data in a controlled setup. The main conclusions is that RGB excels in correspondence estimation, while depth contributes to the pose accuracy if good 3D-3D correspondences are available. Naturally, their combination achieves the overall best performance. We perform an extensive evaluation and an ablation study to analyze and validate the results on several challenging datasets. DPODv2 achieves excellent results on all of them while still remaining fast and scalable independent of the used data modality and the type of training data

References in corpus (12)

• Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift
• Instance Normalization: The Missing Ingredient for Fast Stylization
• Open3D: A Modern Library for 3D Data Processing
• Pix2Pose: Pixel-Wise Coordinate Regression of Objects for 6D Pose Estimation
• Learning Descriptors for Object Recognition and 3D Pose Estimation
• Going Further with Point Pair Features
• 3D Object Instance Recognition and Pose Estimation Using Triplet Loss with Dynamic Margin
• BOP: Benchmark for 6D Object Pose Estimation
• PVN3D: A Deep Point-wise 3D Keypoints Voting Network for 6DoF Pose Estimation
• When Regression Meets Manifold Learning for Object Recognition and Pose Estimation
• Multi-View Object Pose Refinement With Differentiable Renderer
• BlenderProc

Cited by in corpus (5)

• Challenges for Monocular 6D Object Pose Estimation in Robotics
• GDRNPP: A Geometry-guided and Fully Learning-based Object Pose Estimator
• Deep Learning-Based Object Pose Estimation: A Comprehensive Survey
• Diff9D: Diffusion-Based Domain-Generalized Category-Level 9-DoF Object Pose Estimation
• NeRF-Feat: 6D Object Pose Estimation using Feature Rendering