Deep Object Pose Estimation for Semantic Robotic Grasping of Household Objects

Abstract

Using synthetic data for training deep neural networks for robotic manipulation holds the promise of an almost unlimited amount of pre-labeled training data, generated safely out of harm's way. One of the key challenges of synthetic data, to date, has been to bridge the so-called reality gap, so that networks trained on synthetic data operate correctly when exposed to real-world data. We explore the reality gap in the context of 6-DoF pose estimation of known objects from a single RGB image. We show that for this problem the reality gap can be successfully spanned by a simple combination of domain randomized and photorealistic data. Using synthetic data generated in this manner, we introduce a one-shot deep neural network that is able to perform competitively against a state-of-the-art network trained on a combination of real and synthetic data. To our knowledge, this is the first deep network trained only on synthetic data that is able to achieve state-of-the-art performance on 6-DoF object pose estimation. Our network also generalizes better to novel environments including extreme lighting conditions, for which we show qualitative results. Using this network we demonstrate a real-time system estimating object poses with sufficient accuracy for real-world semantic grasping of known household objects in clutter by a real robot.

Conference on Robot Learning (CoRL) 2018

Cited by in corpus (67)

• DeepIM: Deep Iterative Matching for 6D Pose Estimation
• Semantics for Robotic Mapping, Perception and Interaction: A Survey
• A Review of Robot Learning for Manipulation: Challenges, Representations, and Algorithms
• EfficientPose: An efficient, accurate and scalable end-to-end 6D multi object pose estimation approach
• Category-Level Metric Scale Object Shape and Pose Estimation
• PoseRBPF: A Rao-Blackwellized Particle Filter for 6D Object Pose Tracking
• The Best of Both Modes: Separately Leveraging RGB and Depth for Unseen Object Instance Segmentation
• Transferring Dexterous Manipulation from GPU Simulation to a Remote Real-World TriFinger
• Learning RGB-D Feature Embeddings for Unseen Object Instance Segmentation
• REDE: End-to-end Object 6D Pose Robust Estimation Using Differentiable Outliers Elimination
• Robotic Pick-and-Place With Uncertain Object Instance Segmentation and Shape Completion
• DGCM-Net: Dense Geometrical Correspondence Matching Network for Incremental Experience-based Robotic Grasping
• An Annotation Saved is an Annotation Earned: Using Fully Synthetic Training for Object Instance Detection
• MobilePose: Real-Time Pose Estimation for Unseen Objects with Weak Shape Supervision
• Towards Robust Learning-Based Pose Estimation of Noncooperative Spacecraft
• Object SLAM-Based Active Mapping and Robotic Grasping
• Style-transfer GANs for bridging the domain gap in synthetic pose estimator training
• I Like to Move It: 6D Pose Estimation as an Action Decision Process
• Probabilistic orientation estimation with matrix Fisher distributions
• FFB6D: A Full Flow Bidirectional Fusion Network for 6D Pose Estimation
• SimNet: Enabling Robust Unknown Object Manipulation from Pure Synthetic Data via Stereo
• Self-supervised 6D Object Pose Estimation for Robot Manipulation
• Hierarchical Planning for Long-Horizon Manipulation with Geometric and Symbolic Scene Graphs
• Unseen Object Instance Segmentation for Robotic Environments
• 6D Object Pose Regression via Supervised Learning on Point Clouds
• Few-Shot Viewpoint Estimation
• Amodal 3D Reconstruction for Robotic Manipulation via Stability and Connectivity
• Cut-and-Paste Dataset Generation for Balancing Domain Gaps in Object Instance Detection
• NViSII: A Scriptable Tool for Photorealistic Image Generation
• FS-Net: Fast Shape-based Network for Category-Level 6D Object Pose Estimation with Decoupled Rotation Mechanism
• 6D Pose Estimation with Correlation Fusion
• Semantic keypoint-based pose estimation from single RGB frames
• ACR-Pose: Adversarial Canonical Representation Reconstruction Network for Category Level 6D Object Pose Estimation
• Rapid Pose Label Generation through Sparse Representation of Unknown Objects
• se(3)-TrackNet: Data-driven 6D Pose Tracking by Calibrating Image Residuals in Synthetic Domains
• Robotic Task Success Evaluation Under Multi-modal Non-Parametric Object Pose Uncertainty
• Soft-Jig-Driven Assembly Operations
• Computational Ceramicology
• Perception for Autonomous Systems (PAZ)
• DiscoBox: Weakly Supervised Instance Segmentation and Semantic Correspondence from Box Supervision
• 3DP3: 3D Scene Perception via Probabilistic Programming
• Zero in on Shape: A Generic 2D-3D Instance Similarity Metric learned from Synthetic Data
• Implicit 3D Orientation Learning for 6D Object Detection from RGB Images
• Object-Agnostic Suction Grasp Affordance Detection in Dense Cluster Using Self-Supervised Learning.docx
• Contextual Reinforcement Learning of Visuo-tactile Multi-fingered Grasping Policies
• A Mobile Manipulation System for One-Shot Teaching of Complex Tasks in Homes
• Stereo Vision Based Single-Shot 6D Object Pose Estimation for Bin-Picking by a Robot Manipulator
• CloudAAE: Learning 6D Object Pose Regression with On-line Data Synthesis on Point Clouds
• Data-driven 6D Pose Tracking by Calibrating Image Residuals in Synthetic Domains
• P${}^2$GNet: Pose-Guided Point Cloud Generating Networks for 6-DoF Object Pose Estimation
• MELD: Meta-Reinforcement Learning from Images via Latent State Models
• 3D Convolution on RGB-D Point Clouds for Accurate Model-free Object Pose Estimation
• Training Object Detectors on Synthetic Images Containing Reflecting Materials
• Fast Uncertainty Quantification for Deep Object Pose Estimation
• Using Synthetic Data and Deep Networks to Recognize Primitive Shapes for Object Grasping
• Diverse Plausible Shape Completions from Ambiguous Depth Images
• Hardware Acceleration of Monte-Carlo Sampling for Energy Efficient Robust Robot Manipulation
• Differentiable Nonparametric Belief Propagation
• DeepPanoContext: Panoramic 3D Scene Understanding with Holistic Scene Context Graph and Relation-based Optimization
• Manipulation-Oriented Object Perception in Clutter through Affordance Coordinate Frames
• A Dynamic Keypoints Selection Network for 6DoF Pose Estimation
• Object Detection in the Context of Mobile Augmented Reality
• Implicit Shape Model Trees: Recognition of 3-D Indoor Scenes and Prediction of Object Poses for Mobile Robots
• BIDCD -- Bosch Industrial Depth Completion Dataset
• PAMTRI: Pose-Aware Multi-Task Learning for Vehicle Re-Identification Using Highly Randomized Synthetic Data
• VideoPose: Estimating 6D object pose from videos
• Generating Annotated Training Data for 6D Object Pose Estimation in Operational Environments with Minimal User Interaction