DGCM-Net: Dense Geometrical Correspondence Matching Network for Incremental Experience-based Robotic Grasping

Abstract

This article presents a method for grasping novel objects by learning from experience. Successful attempts are remembered and then used to guide future grasps such that more reliable grasping is achieved over time. To generalise the learned experience to unseen objects, we introduce the dense geometric correspondence matching network (DGCM-Net). This applies metric learning to encode objects with similar geometry nearby in feature space. Retrieving relevant experience for an unseen object is thus a nearest neighbour search with the encoded feature maps. DGCM-Net also reconstructs 3D-3D correspondences using the view-dependent normalised object coordinate space to transform grasp configurations from retrieved samples to unseen objects. In comparison to baseline methods, our approach achieves an equivalent grasp success rate. However, the baselines are significantly improved when fusing the knowledge from experience with their grasp proposal strategy. Offline experiments with a grasping dataset highlight the capability to generalise within and between object classes as well as to improve success rate over time from increasing experience. Lastly, by learning task-relevant grasps, our approach can prioritise grasps that enable the functional use of objects.

arXiv commentPublished in the journal "Frontiers in Robotics and AI"

References in corpus (10)

• Adam: A Method for Stochastic Optimization
• Data-Driven Grasp Synthesis - A Survey
• QT-Opt: Scalable Deep Reinforcement Learning for Vision-Based Robotic Manipulation
• Dex-Net 2.0: Deep Learning to Plan Robust Grasps with Synthetic Point Clouds and Analytic Grasp Metrics
• Deep Object Pose Estimation for Semantic Robotic Grasping of Household Objects
• Dense Object Nets: Learning Dense Visual Object Descriptors By and For Robotic Manipulation
• Grasp2Vec: Learning Object Representations from Self-Supervised Grasping
• kPAM: KeyPoint Affordances for Category-Level Robotic Manipulation
• End-to-End Learning of Semantic Grasping
• Global Search with Bernoulli Alternation Kernel for Task-oriented Grasping Informed by Simulation

Cited by in corpus (4)

• Vision-based Robotic Grasping From Object Localization, Object Pose Estimation to Grasp Estimation for Parallel Grippers: A Review
• Challenges for Monocular 6D Object Pose Estimation in Robotics
• RGB Matters: Learning 7-DoF Grasp Poses on Monocular RGBD Images
• Fully Self-Supervised Class Awareness in Dense Object Descriptors