Towards Interactive Reinforcement Learning with Intrinsic Feedback

Abstract

Reinforcement learning (RL) and brain-computer interfaces (BCI) have experienced significant growth over the past decade. With rising interest in human-in-the-loop (HITL), incorporating human input with RL algorithms has given rise to the sub-field of interactive RL. Adjacently, the field of BCI has long been interested in extracting informative brain signals from neural activity for use in human-computer interactions. A key link between these fields lies in the interpretation of neural activity as feedback such that interactive RL approaches can be employed. We denote this new and emerging medium of feedback as intrinsic feedback. Despite intrinsic feedback's ability to be conveyed automatically and even unconsciously, proper exploration surrounding this key link has largely gone unaddressed by both communities. Thus, to help facilitate a deeper understanding and a more effective utilization, we provide a tutorial-style review covering the motivations, approaches, and open problems of intrinsic feedback and its foundational concepts.

arXiv commentName change and vast rewrites of the paper

References in corpus (14)

• EEGNet: A Compact Convolutional Network for EEG-based Brain-Computer Interfaces
• Training language models to follow instructions with human feedback
• Dota 2 with Large Scale Deep Reinforcement Learning
• Artificial Intelligence, Values and Alignment
• Evidential Deep Learning to Quantify Classification Uncertainty
• Beyond Product Quantization: Deep Progressive Quantization for Image Retrieval
• Deep Recurrent Quantization for Generating Sequential Binary Codes
• EEG based Emotion Recognition: A Tutorial and Review
• Scalable agent alignment via reward modeling: a research direction
• DQN-TAMER: Human-in-the-Loop Reinforcement Learning with Intractable Feedback
• Alternative Modes of Interaction in Proximal Human-in-the-Loop Operation of Robots
• A Survey on Interactive Reinforcement Learning: Design Principles and Open Challenges
• Widening the Pipeline in Human-Guided Reinforcement Learning with Explanation and Context-Aware Data Augmentation
• Training Value-Aligned Reinforcement Learning Agents Using a Normative Prior