Reinforcement Learning for Vision-based Object Manipulation with Non-parametric Policy and Action Primitives

Abstract

The object manipulation is a crucial ability for a service robot, but it is hard to solve with reinforcement learning due to some reasons such as sample efficiency. In this paper, to tackle this object manipulation, we propose a novel framework, AP-NPQL (Non-Parametric Q Learning with Action Primitives), that can efficiently solve the object manipulation with visual input and sparse reward, by utilizing a non-parametric policy for reinforcement learning and appropriate behavior prior for the object manipulation. We evaluate the efficiency and the performance of the proposed AP-NPQL for four object manipulation tasks on simulation (pushing plate, stacking box, flipping cup, and picking and placing plate), and it turns out that our AP-NPQL outperforms the state-of-the-art algorithms based on parametric policy and behavior prior in terms of learning time and task success rate. We also successfully transfer and validate the learned policy of the plate pick-and-place task to the real robot in a sim-to-real manner.

References in corpus (17)

• Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actor
• Addressing Function Approximation Error in Actor-Critic Methods
• Learning agile and dynamic motor skills for legged robots
• QT-Opt: Scalable Deep Reinforcement Learning for Vision-Based Robotic Manipulation
• Solving Rubik's Cube with a Robot Hand
• Reinforcement Learning and Control as Probabilistic Inference: Tutorial and Review
• Distributed Distributional Deterministic Policy Gradients
• Off-Policy Deep Reinforcement Learning without Exploration
• Learning by Playing - Solving Sparse Reward Tasks from Scratch
• Behavior Regularized Offline Reinforcement Learning
• Maximum a Posteriori Policy Optimisation
• Critic Regularized Regression
• Making Efficient Use of Demonstrations to Solve Hard Exploration Problems
• Keep Doing What Worked: Behavioral Modelling Priors for Offline Reinforcement Learning
• Learning RGB-D Feature Embeddings for Unseen Object Instance Segmentation
• DSAC: Distributional Soft Actor-Critic for Risk-Sensitive Reinforcement Learning
• An operator view of policy gradient methods