Interactive Visualization for Debugging RL

Abstract

Visualization tools for supervised learning allow users to interpret, introspect, and gain an intuition for the successes and failures of their models. While reinforcement learning practitioners ask many of the same questions, existing tools are not applicable to the RL setting as these tools address challenges typically found in the supervised learning regime. In this work, we design and implement an interactive visualization tool for debugging and interpreting RL algorithms. Our system addresses many features missing from previous tools such as (1) tools for supervised learning often are not interactive; (2) while debugging RL policies researchers use state representations that are different from those seen by the agent; (3) a framework designed to make the debugging RL policies more conducive. We provide an example workflow of how this system could be used, along with ideas for future extensions.

Builds on preliminary work presented at ICML 2020 (WHI) arXiv:2007.05577. An interactive demo of the system can be at https://tinyurl.com/y5gv5t4m

References in corpus (10)

• Playing Atari with Deep Reinforcement Learning
• Towards A Rigorous Science of Interpretable Machine Learning
• Addressing Function Approximation Error in Actor-Critic Methods
• High-Dimensional Continuous Control Using Generalized Advantage Estimation
• Understanding Neural Networks Through Deep Visualization
• D4RL: Datasets for Deep Data-Driven Reinforcement Learning
• A Deeper Look at Experience Replay
• Visualizing and Understanding Atari Agents
• Reinforcement Learning in Healthcare: A Survey
• Vizarel: A System to Help Better Understand RL Agents

Cited by in corpus (2)

• d3rlpy: An Offline Deep Reinforcement Learning Library
• Why? Why not? When? Visual Explanations of Agent Behavior in Reinforcement Learning