Learning Force Control for Contact-rich Manipulation Tasks with Rigid Position-controlled Robots

Abstract

Reinforcement Learning (RL) methods have been proven successful in solving manipulation tasks autonomously. However, RL is still not widely adopted on real robotic systems because working with real hardware entails additional challenges, especially when using rigid position-controlled manipulators. These challenges include the need for a robust controller to avoid undesired behavior, that risk damaging the robot and its environment, and constant supervision from a human operator. The main contributions of this work are, first, we proposed a learning-based force control framework combining RL techniques with traditional force control. Within said control scheme, we implemented two different conventional approaches to achieve force control with position-controlled robots; one is a modified parallel position/force control, and the other is an admittance control. Secondly, we empirically study both control schemes when used as the action space of the RL agent. Thirdly, we developed a fail-safe mechanism for safely training an RL agent on manipulation tasks using a real rigid robot manipulator. The proposed methods are validated on simulation and a real robot, an UR3 e-series robotic arm.

8 pages, 9 figures, version accepted for IROS RA-L 2020, for associated video file, see https://youtu.be/4wdIhQxD6cA

References in corpus (6)

• Continuous control with deep reinforcement learning
• Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actor
• End-to-End Training of Deep Visuomotor Policies
• QT-Opt: Scalable Deep Reinforcement Learning for Vision-Based Robotic Manipulation
• A Review of Robot Learning for Manipulation: Challenges, Representations, and Algorithms
• Benchmarking Reinforcement Learning Algorithms on Real-World Robots

Cited by in corpus (13)

• Variable Compliance Control for Robotic Peg-in-Hole Assembly: A Deep Reinforcement Learning Approach
• A Survey of Robot Manipulation in Contact
• Deep Model Predictive Variable Impedance Control
• Visual Spatial Attention and Proprioceptive Data-Driven Reinforcement Learning for Robust Peg-in-Hole Task Under Variable Conditions
• Soft and Rigid Object Grasping With Cross-Structure Hand Using Bilateral Control-Based Imitation Learning
• On the Role of the Action Space in Robot Manipulation Learning and Sim-to-Real Transfer
• Controlling Pivoting Gait using Graph Model Predictive Control
• A Contact-Safe Reinforcement Learning Framework for Contact-Rich Robot Manipulation
• Robotic Imitation of Human Assembly Skills Using Hybrid Trajectory and Force Learning
• Belief-Grounded Networks for Accelerated Robot Learning under Partial Observability
• SCAPE: Learning Stiffness Control from Augmented Position Control Experiences
• Uncertainty-aware Contact-safe Model-based Reinforcement Learning
• Learning Robotic Manipulation Skills Using an Adaptive Force-Impedance Action Space