DeepIM: Deep Iterative Matching for 6D Pose Estimation

Abstract

Estimating the 6D pose of objects from images is an important problem in various applications such as robot manipulation and virtual reality. While direct regression of images to object poses has limited accuracy, matching rendered images of an object against the observed image can produce accurate results. In this work, we propose a novel deep neural network for 6D pose matching named DeepIM. Given an initial pose estimation, our network is able to iteratively refine the pose by matching the rendered image against the observed image. The network is trained to predict a relative pose transformation using an untangled representation of 3D location and 3D orientation and an iterative training process. Experiments on two commonly used benchmarks for 6D pose estimation demonstrate that DeepIM achieves large improvements over state-of-the-art methods. We furthermore show that DeepIM is able to match previously unseen objects.

submitted to IJCV, update results on YCB_Video, add depth-based results

References in corpus (15)

• Very Deep Convolutional Networks for Large-Scale Image Recognition
• SSD: Single Shot MultiBox Detector
• Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks
• PointNet: Deep Learning on Point Sets for 3D Classification and Segmentation
• Fully Convolutional Networks for Semantic Segmentation
• Go-ICP: A Globally Optimal Solution to 3D ICP Point-Set Registration
• DeepVO: Towards End-to-End Visual Odometry with Deep Recurrent Convolutional Neural Networks
• Deep Object Pose Estimation for Semantic Robotic Grasping of Household Objects
• PoseCNN: A Convolutional Neural Network for 6D Object Pose Estimation in Cluttered Scenes
• Going Further with Point Pair Features
• DenseFusion: 6D Object Pose Estimation by Iterative Dense Fusion
• Deep 6-DOF Tracking
• LS-VO: Learning Dense Optical Subspace for Robust Visual Odometry Estimation
• PoseRBPF: A Rao-Blackwellized Particle Filter for 6D Object Pose Tracking
• Real-Time Seamless Single Shot 6D Object Pose Prediction

Cited by in corpus (25)

• DenseFusion: 6D Object Pose Estimation by Iterative Dense Fusion
• SRT3D: A Sparse Region-Based 3D Object Tracking Approach for the Real World
• Vision-driven Compliant Manipulation for Reliable, High-Precision Assembly Tasks
• Occlusion-Aware Self-Supervised Monocular 6D Object Pose Estimation
• Tac2Pose: Tactile Object Pose Estimation from the First Touch
• Pose from Shape: Deep Pose Estimation for Arbitrary 3D Objects
• 3D Object Detection and Pose Estimation of Unseen Objects in Color Images with Local Surface Embeddings
• GDRNPP: A Geometry-guided and Fully Learning-based Object Pose Estimator
• 6DoF Object Pose Estimation via Differentiable Proxy Voting Loss
• Neural Object Learning for 6D Pose Estimation Using a Few Cluttered Images
• Pose Estimation for Non-Cooperative Rendezvous Using Neural Networks
• Accurate 6D Object Pose Estimation by Pose Conditioned Mesh Reconstruction
• 6-PACK: Category-level 6D Pose Tracker with Anchor-Based Keypoints
• Differentiable Rendering with Perturbed Optimizers
• Squeezed Deep 6DoF Object Detection Using Knowledge Distillation
• Probabilistic Category-Level Pose Estimation via Segmentation and Predicted-Shape Priors
• DTF-Net: Category-Level Pose Estimation and Shape Reconstruction via Deformable Template Field
• AI-Based Multi-Object Relative State Estimation with Self-Calibration Capabilities
• Data-driven 6D Pose Tracking by Calibrating Image Residuals in Synthetic Domains
• Robust 6DoF Pose Tracking Considering Contour and Interior Correspondence Uncertainty for AR Assembly Guidance
• Occlusion Resistant Object Rotation Regression from Point Cloud Segments
• Physics-Based Rigid Body Object Tracking and Friction Filtering From RGB-D Videos
• Lessons Learned Developing an Assembly System for WRS 2020 Assembly Challenge
• Veriserum: A dual-plane fluoroscopic dataset with knee implant phantoms for deep learning in medical imaging
• Deep Measurement Updates for Bayes Filters