Learning Task-Oriented Grasping for Tool Manipulation from Simulated Self-Supervision

Abstract

Tool manipulation is vital for facilitating robots to complete challenging task goals. It requires reasoning about the desired effect of the task and thus properly grasping and manipulating the tool to achieve the task. Task-agnostic grasping optimizes for grasp robustness while ignoring crucial task-specific constraints. In this paper, we propose the Task-Oriented Grasping Network (TOG-Net) to jointly optimize both task-oriented grasping of a tool and the manipulation policy for that tool. The training process of the model is based on large-scale simulated self-supervision with procedurally generated tool objects. We perform both simulated and real-world experiments on two tool-based manipulation tasks: sweeping and hammering. Our model achieves overall 71.1% task success rate for sweeping and 80.0% task success rate for hammering. Supplementary material is available at: bit.ly/task-oriented-grasp

arXiv commentRSS 2018

References in corpus (5)

• Dex-Net 2.0: Deep Learning to Plan Robust Grasps with Synthetic Point Clouds and Analytic Grasp Metrics
• Learning a visuomotor controller for real world robotic grasping using simulated depth images
• End-to-End Learning of Semantic Grasping
• AffordanceNet: An End-to-End Deep Learning Approach for Object Affordance Detection
• Grasping for a Purpose: Using Task Goals for Efficient Manipulation Planning

Cited by in corpus (7)

• Rapid trial-and-error learning with simulation supports flexible tool use and physical reasoning
• Mechanical Search: Multi-Step Retrieval of a Target Object Occluded by Clutter
• RoboTurk: A Crowdsourcing Platform for Robotic Skill Learning through Imitation
• How to Close Sim-Real Gap? Transfer with Segmentation!
• Reasoning and Generalization in RL: A Tool Use Perspective
• Imagine That! Leveraging Emergent Affordances for 3D Tool Synthesis
• Deictic Image Maps: An Abstraction For Learning Pose Invariant Manipulation Policies