Visual Perspective Taking for Opponent Behavior Modeling

Abstract

In order to engage in complex social interaction, humans learn at a young age to infer what others see and cannot see from a different point-of-view, and learn to predict others' plans and behaviors. These abilities have been mostly lacking in robots, sometimes making them appear awkward and socially inept. Here we propose an end-to-end long-term visual prediction framework for robots to begin to acquire both these critical cognitive skills, known as Visual Perspective Taking (VPT) and Theory of Behavior (TOB). We demonstrate our approach in the context of visual hide-and-seek - a game that represents a cognitive milestone in human development. Unlike traditional visual predictive model that generates new frames from immediate past frames, our agent can directly predict to multiple future timestamps (25s), extrapolating by 175% beyond the training horizon. We suggest that visual behavior modeling and perspective taking skills will play a critical role in the ability of physical robots to fully integrate into real-world multi-agent activities. Our website is at http://www.cs.columbia.edu/~bchen/vpttob/.

arXiv commentICRA 2021. Website: http://www.cs.columbia.edu/~bchen/vpttob/

References in corpus (16)

• Adam: A Method for Stochastic Optimization
• Learning to Communicate with Deep Multi-Agent Reinforcement Learning
• Unity: A General Platform for Intelligent Agents
• Autonomous Agents Modelling Other Agents: A Comprehensive Survey and Open Problems
• Emergent Tool Use From Multi-Agent Autocurricula
• Learning Latent Dynamics for Planning from Pixels
• Visual Foresight: Model-Based Deep Reinforcement Learning for Vision-Based Robotic Control
• Dream to Control: Learning Behaviors by Latent Imagination
• Recurrent Environment Simulators
• Spatial Action Maps for Mobile Manipulation
• Intention-Net: Integrating Planning and Deep Learning for Goal-Directed Autonomous Navigation
• Learning and Querying Fast Generative Models for Reinforcement Learning
• Emergent Communication through Negotiation
• FollowNet: Robot Navigation by Following Natural Language Directions with Deep Reinforcement Learning
• A Deep Learning Approach for Joint Video Frame and Reward Prediction in Atari Games
• VisuoSpatial Foresight for Multi-Step, Multi-Task Fabric Manipulation