Pix2Pose: Pixel-Wise Coordinate Regression of Objects for 6D Pose Estimation

Abstract

Estimating the 6D pose of objects using only RGB images remains challenging because of problems such as occlusion and symmetries. It is also difficult to construct 3D models with precise texture without expert knowledge or specialized scanning devices. To address these problems, we propose a novel pose estimation method, Pix2Pose, that predicts the 3D coordinates of each object pixel without textured models. An auto-encoder architecture is designed to estimate the 3D coordinates and expected errors per pixel. These pixel-wise predictions are then used in multiple stages to form 2D-3D correspondences to directly compute poses with the PnP algorithm with RANSAC iterations. Our method is robust to occlusion by leveraging recent achievements in generative adversarial training to precisely recover occluded parts. Furthermore, a novel loss function, the transformer loss, is proposed to handle symmetric objects by guiding predictions to the closest symmetric pose. Evaluations on three different benchmark datasets containing symmetric and occluded objects show our method outperforms the state of the art using only RGB images.

Accepted at ICCV 2019 (Oral)

References in corpus (1)

• Benchmarking in Manipulation Research: The YCB Object and Model Set and Benchmarking Protocols

Cited by in corpus (27)

• Vision-based Robotic Grasping From Object Localization, Object Pose Estimation to Grasp Estimation for Parallel Grippers: A Review
• A Survey on Deep Learning-Based Monocular Spacecraft Pose Estimation: Current State, Limitations and Prospects
• Occlusion-Aware Self-Supervised Monocular 6D Object Pose Estimation
• DPODv2: Dense Correspondence-Based 6 DoF Pose Estimation
• Category-Level Metric Scale Object Shape and Pose Estimation
• REDE: End-to-end Object 6D Pose Robust Estimation Using Differentiable Outliers Elimination
• Towards 3D Face Reconstruction in Perspective Projection: Estimating 6DoF Face Pose from Monocular Image
• 3D Object Detection and Pose Estimation of Unseen Objects in Color Images with Local Surface Embeddings
• Multi-View Object Pose Refinement With Differentiable Renderer
• GDRNPP: A Geometry-guided and Fully Learning-based Object Pose Estimator
• Deep Learning-Based Object Pose Estimation: A Comprehensive Survey
• DISP6D: Disentangled Implicit Shape and Pose Learning for Scalable 6D Pose Estimation
• Object-Based Visual Camera Pose Estimation From Ellipsoidal Model and 3D-Aware Ellipse Prediction
• ViHOPE: Visuotactile In-Hand Object 6D Pose Estimation with Shape Completion
• Neural Object Learning for 6D Pose Estimation Using a Few Cluttered Images
• KVN: Keypoints Voting Network with Differentiable RANSAC for Stereo Pose Estimation
• DTF-Net: Category-Level Pose Estimation and Shape Reconstruction via Deformable Template Field
• Imitrob: Imitation Learning Dataset for Training and Evaluating 6D Object Pose Estimators
• FocalPose++: Focal Length and Object Pose Estimation via Render and Compare
• NeRF-Feat: 6D Object Pose Estimation using Feature Rendering
• Lost & Found: Tracking Changes from Egocentric Observations in 3D Dynamic Scene Graphs
• In-hand manipulation planning using human motion dictionary
• Monocular pose estimation of articulated open surgery tools -- in the wild
• Glissando-Net: Deep sinGLe vIew category level poSe eStimation ANd 3D recOnstruction
• Pose Estimation of Buried Deep-Sea Objects using 3D Vision Deep Learning Models
• L6DNet: Light 6 DoF Network for Robust and Precise Object Pose Estimation with Small Datasets
• WAPR: A Foundation Model for Wide-Angle Refinement in Unseen Object Pose Estimation