DenseFusion: 6D Object Pose Estimation by Iterative Dense Fusion

Abstract

A key technical challenge in performing 6D object pose estimation from RGB-D image is to fully leverage the two complementary data sources. Prior works either extract information from the RGB image and depth separately or use costly post-processing steps, limiting their performances in highly cluttered scenes and real-time applications. In this work, we present DenseFusion, a generic framework for estimating 6D pose of a set of known objects from RGB-D images. DenseFusion is a heterogeneous architecture that processes the two data sources individually and uses a novel dense fusion network to extract pixel-wise dense feature embedding, from which the pose is estimated. Furthermore, we integrate an end-to-end iterative pose refinement procedure that further improves the pose estimation while achieving near real-time inference. Our experiments show that our method outperforms state-of-the-art approaches in two datasets, YCB-Video and LineMOD. We also deploy our proposed method to a real robot to grasp and manipulate objects based on the estimated pose.

References in corpus (6)

• PointNet: Deep Learning on Point Sets for 3D Classification and Segmentation
• VoxelNet: End-to-End Learning for Point Cloud Based 3D Object Detection
• DeepIM: Deep Iterative Matching for 6D Pose Estimation
• Rotational Subgroup Voting and Pose Clustering for Robust 3D Object Recognition
• 6-DoF Object Pose from Semantic Keypoints
• A Unified Framework for Multi-View Multi-Class Object Pose Estimation

Cited by in corpus (16)

• DeepIM: Deep Iterative Matching for 6D Pose Estimation
• Dex-NeRF: Using a Neural Radiance Field to Grasp Transparent Objects
• PoseRBPF: A Rao-Blackwellized Particle Filter for 6D Object Pose Tracking
• Robust, Occlusion-aware Pose Estimation for Objects Grasped by Adaptive Hands
• PVN3D: A Deep Point-wise 3D Keypoints Voting Network for 6DoF Pose Estimation
• Data-Efficient Learning for Sim-to-Real Robotic Grasping using Deep Point Cloud Prediction Networks
• G2L-Net: Global to Local Network for Real-time 6D Pose Estimation with Embedding Vector Features
• GRIP: Generative Robust Inference and Perception for Semantic Robot Manipulation in Adversarial Environments
• Robust 6D Object Pose Estimation by Learning RGB-D Features
• Contextual Reinforcement Learning of Visuo-tactile Multi-fingered Grasping Policies
• Data-driven 6D Pose Tracking by Calibrating Image Residuals in Synthetic Domains
• A Review on Object Pose Recovery: from 3D Bounding Box Detectors to Full 6D Pose Estimators
• GarmentNets: Category-Level Pose Estimation for Garments via Canonical Space Shape Completion
• Pose Estimation for Texture-less Shiny Objects in a Single RGB Image Using Synthetic Training Data
• EnforceNet: Monocular Camera Localization in Large Scale Indoor Sparse LiDAR Point Cloud
• GeoFusion: Geometric Consistency informed Scene Estimation in Dense Clutter