Tactile Tool Manipulation

Abstract

Humans can effortlessly perform very complex, dexterous manipulation tasks by reacting to sensor observations. In contrast, robots can not perform reactive manipulation and they mostly operate in open-loop while interacting with their environment. Consequently, the current manipulation algorithms either are inefficient in performance or can only work in highly structured environments. In this paper, we present closed-loop control of a complex manipulation task where a robot uses a tool to interact with objects. Manipulation using a tool leads to complex kinematics and contact constraints that need to be satisfied for generating feasible manipulation trajectories. We first present an open-loop controller design using Non-Linear Programming (NLP) that satisfies these constraints. In order to design a closed-loop controller, we present a pose estimator of objects and tools using tactile sensors. Using our tactile estimator, we design a closed-loop controller based on Model Predictive Control (MPC). The proposed algorithm is verified using a 6 DoF manipulator on tasks using a variety of objects and tools. We verify that our closed-loop controller can successfully perform tool manipulation under several unexpected contacts. Video summarizing this work and hardware experiments are found https://youtu.be/VsClK04qDhk.

Accepted to ICRA2023. Video: https://youtu.be/VsClK04qDhk

References in corpus (4)

• Simultaneous Contact-Rich Grasping and Locomotion via Distributed Optimization Enabling Free-Climbing for Multi-Limbed Robots
• Robust Pivoting: Exploiting Frictional Stability Using Bilevel Optimization
• Covariance Steering for Uncertain Contact-rich Systems
• Chance-Constrained Optimization in Contact-Rich Systems for Robust Manipulation

Cited by in corpus (5)

• A Survey of Embodied Learning for Object-Centric Robotic Manipulation
• Robust Pivoting Manipulation using Contact Implicit Bilevel Optimization
• Hierarchical Contact-Rich Trajectory Optimization for Multi-Modal Manipulation using Tight Convex Relaxations
• Trajectory Optimization for In-Hand Manipulation with Tactile Force Control
• TEXterity -- Tactile Extrinsic deXterity: Simultaneous Tactile Estimation and Control for Extrinsic Dexterity