Psychlab: A Psychology Laboratory for Deep Reinforcement Learning Agents

Abstract

Psychlab is a simulated psychology laboratory inside the first-person 3D game world of DeepMind Lab (Beattie et al. 2016). Psychlab enables implementations of classical laboratory psychological experiments so that they work with both human and artificial agents. Psychlab has a simple and flexible API that enables users to easily create their own tasks. As examples, we are releasing Psychlab implementations of several classical experimental paradigms including visual search, change detection, random dot motion discrimination, and multiple object tracking. We also contribute a study of the visual psychophysics of a specific state-of-the-art deep reinforcement learning agent: UNREAL (Jaderberg et al. 2016). This study leads to the surprising conclusion that UNREAL learns more quickly about larger target stimuli than it does about smaller stimuli. In turn, this insight motivates a specific improvement in the form of a simple model of foveal vision that turns out to significantly boost UNREAL's performance, both on Psychlab tasks, and on standard DeepMind Lab tasks. By open-sourcing Psychlab we hope to facilitate a range of future such studies that simultaneously advance deep reinforcement learning and improve its links with cognitive science.

arXiv comment28 pages, 11 figures

References in corpus (10)

• Neural Machine Translation by Jointly Learning to Align and Translate
• DRAW: A Recurrent Neural Network For Image Generation
• Multiple Object Recognition with Visual Attention
• RL${}^2$: Fast Reinforcement Learning via Slow Reinforcement Learning
• Learning to reinforcement learn
• Grounded Language Learning in a Simulated 3D World
• Recurrent Environment Simulators
• Gated-Attention Architectures for Task-Oriented Language Grounding
• Cognitive Psychology for Deep Neural Networks: A Shape Bias Case Study
• Understanding Early Word Learning in Situated Artificial Agents

Cited by in corpus (15)

• Unity: A General Platform for Intelligent Agents
• ViZDoom Competitions: Playing Doom from Pixels
• Autocurricula and the Emergence of Innovation from Social Interaction: A Manifesto for Multi-Agent Intelligence Research
• A neural network walks into a lab: towards using deep nets as models for human behavior
• Kickstarting Deep Reinforcement Learning
• Re-evaluating Evaluation
• Closing the AI Knowledge Gap
• Artificial mental phenomena: Psychophysics as a framework to detect perception biases in AI models
• Going Beyond Linear Transformers with Recurrent Fast Weight Programmers
• Evaluating Curriculum Learning Strategies in Neural Combinatorial Optimization
• Arena: A General Evaluation Platform and Building Toolkit for Multi-Agent Intelligence
• Using Unity to Help Solve Intelligence
• CUDA-Optimized real-time rendering of a Foveated Visual System
• The Emergence of Abstract and Episodic Neurons in Episodic Meta-RL
• Psychophysical Evaluation of Deep Re-Identification Models