Multi-modal Transfer Learning for Grasping Transparent and Specular Objects
arXiv:2006.00028 · doi:10.1109/LRA.2020.2974686
Abstract
State-of-the-art object grasping methods rely on depth sensing to plan robust grasps, but commercially available depth sensors fail to detect transparent and specular objects. To improve grasping performance on such objects, we introduce a method for learning a multi-modal perception model by bootstrapping from an existing uni-modal model. This transfer learning approach requires only a pre-existing uni-modal grasping model and paired multi-modal image data for training, foregoing the need for ground-truth grasp success labels nor real grasp attempts. Our experiments demonstrate that our approach is able to reliably grasp transparent and reflective objects. Video and supplementary material are available at https://sites.google.com/view/transparent-specular-grasping.
RA-L with presentation at ICRA 2020
References in corpus (7)
- Data-Driven Grasp Synthesis - A Survey
- QT-Opt: Scalable Deep Reinforcement Learning for Vision-Based Robotic Manipulation
- Benchmarking in Manipulation Research: The YCB Object and Model Set and Benchmarking Protocols
- Dex-Net 2.0: Deep Learning to Plan Robust Grasps with Synthetic Point Clouds and Analytic Grasp Metrics
- Learning a visuomotor controller for real world robotic grasping using simulated depth images
- Cross-Modal Attentional Context Learning for RGB-D Object Detection
- Jacquard: A Large Scale Dataset for Robotic Grasp Detection
Cited by in corpus (6)
- Deep Transfer Learning for Industrial Automation: A Review and Discussion of New Techniques for Data-Driven Machine Learning
- Visual-tactile Fusion for Transparent Object Grasping in Complex Backgrounds
- RFTrans: Leveraging Refractive Flow of Transparent Objects for Surface Normal Estimation and Manipulation
- Fuzzy-Depth Objects Grasping Based on FSG Algorithm and a Soft Robotic Hand
- Robotic Manipulation Framework Based on Semantic Keypoints for Packing Shoes of Different Sizes, Shapes, and Softness
- Mirror3D: Depth Refinement for Mirror Surfaces