Agent-Agnostic Human-in-the-Loop Reinforcement Learning

Abstract

Providing Reinforcement Learning agents with expert advice can dramatically improve various aspects of learning. Prior work has developed teaching protocols that enable agents to learn efficiently in complex environments; many of these methods tailor the teacher's guidance to agents with a particular representation or underlying learning scheme, offering effective but specialized teaching procedures. In this work, we explore protocol programs, an agent-agnostic schema for Human-in-the-Loop Reinforcement Learning. Our goal is to incorporate the beneficial properties of a human teacher into Reinforcement Learning without making strong assumptions about the inner workings of the agent. We show how to represent existing approaches such as action pruning, reward shaping, and training in simulation as special cases of our schema and conduct preliminary experiments on simple domains.

Presented at the NIPS Workshop on the Future of Interactive Learning Machines, 2016

References in corpus (6)

• Benchmarking Deep Reinforcement Learning for Continuous Control
• Near Optimal Behavior via Approximate State Abstraction
• Model Reduction Techniques for Computing Approximately Optimal Solutions for Markov Decision Processes
• Combating Reinforcement Learning's Sisyphean Curse with Intrinsic Fear
• Theoretically-Grounded Policy Advice from Multiple Teachers in Reinforcement Learning Settings with Applications to Negative Transfer
• Domain-Independent Optimistic Initialization for Reinforcement Learning

Cited by in corpus (15)

• Scalable agent alignment via reward modeling: a research direction
• Reinforcement Learning in Healthcare: A Survey
• AI Research Considerations for Human Existential Safety (ARCHES)
• A Survey on Dialog Management: Recent Advances and Challenges
• A Survey on Interactive Reinforcement Learning: Design Principles and Open Challenges
• Deep Active Learning for Dialogue Generation
• Novelty Search in Representational Space for Sample Efficient Exploration
• Accelerating Safe Reinforcement Learning with Constraint-mismatched Policies
• Safe Driving via Expert Guided Policy Optimization
• Learning Shaping Strategies in Human-in-the-loop Interactive Reinforcement Learning
• Battlesnake Challenge: A Multi-agent Reinforcement Learning Playground with Human-in-the-loop
• Curiosity Killed or Incapacitated the Cat and the Asymptotically Optimal Agent
• Teachable Reinforcement Learning via Advice Distillation
• Towards an unanimous international regulatory body for responsible use of Artificial Intelligence [UIRB-AI]
• A Multi-step and Resilient Predictive Q-learning Algorithm for IoT with Human Operators in the Loop: A Case Study in Water Supply Networks