Interactive Policy Learning through Confidence-Based Autonomy

Abstract

We present Confidence-Based Autonomy (CBA), an interactive algorithm for policy learning from demonstration. The CBA algorithm consists of two components which take advantage of the complimentary abilities of humans and computer agents. The first component, Confident Execution, enables the agent to identify states in which demonstration is required, to request a demonstration from the human teacher and to learn a policy based on the acquired data. The algorithm selects demonstrations based on a measure of action selection confidence, and our results show that using Confident Execution the agent requires fewer demonstrations to learn the policy than when demonstrations are selected by a human teacher. The second algorithmic component, Corrective Demonstration, enables the teacher to correct any mistakes made by the agent through additional demonstrations in order to improve the policy and future task performance. CBA and its individual components are compared and evaluated in a complex simulated driving domain. The complete CBA algorithm results in the best overall learning performance, successfully reproducing the behavior of the teacher while balancing the tradeoff between number of demonstrations and number of incorrect actions during learning.

References in corpus (2)

• Interactive Policy Learning through Confidence-Based Autonomy
• Accelerating Reinforcement Learning through Implicit Imitation

Cited by in corpus (23)

• Active Learning of Inverse Models with Intrinsically Motivated Goal Exploration in Robots
• An Algorithmic Perspective on Imitation Learning
• Interactive Policy Learning through Confidence-Based Autonomy
• A Review of Robot Learning for Manipulation: Challenges, Representations, and Algorithms
• Active choice of teachers, learning strategies and goals for a socially guided intrinsic motivation learner
• Socially Guided Intrinsic Motivation for Robot Learning of Motor Skills
• Learning User's confidence for active learning
• Active Imitation Learning via Reduction to I.I.D. Active Learning
• Intrinsically Motivated Open-Ended Multi-Task Learning Using Transfer Learning to Discover Task Hierarchy
• Human-in-the-Loop Imitation Learning using Remote Teleoperation
• CrowdTransfer: Enabling Crowd Knowledge Transfer in AIoT Community
• Improving Robot-Centric Learning from Demonstration via Personalized Embeddings
• SAFARI: Safe and Active Robot Imitation Learning with Imagination
• Confidence-Based Skill Reproduction Through Perturbation Analysis
• Improving Competence for Reliable Autonomy
• An Imitation Game for Learning Semantic Parsers from User Interaction
• Multi-class Generalized Binary Search for Active Inverse Reinforcement Learning
• Human-In-The-Loop Document Layout Analysis
• Learning Gaussian Policies from Corrective Human Feedback
• Cooperative Queuing Policies for Effective Human-Multi-Robot Interaction
• Investigating the Effects of Robot Engagement Communication on Learning from Demonstration
• Interactive Imitation Learning in State-Space
• Active Hierarchical Imitation and Reinforcement Learning