Affordance as general value function: A computational model
arXiv:2010.14289
Abstract
General value functions (GVFs) in the reinforcement learning (RL) literature are long-term predictive summaries of the outcomes of agents following specific policies in the environment. Affordances as perceived action possibilities with specific valence may be cast into predicted policy-relative goodness and modelled as GVFs. A systematic explication of this connection shows that GVFs and especially their deep learning embodiments (1) realize affordance prediction as a form of direct perception, (2) illuminate the fundamental connection between action and perception in affordance, and (3) offer a scalable way to learn affordances using RL methods. Through an extensive review of existing literature on GVF applications and representative affordance research in robotics, we demonstrate that GVFs provide the right framework for learning affordances in real-world applications. In addition, we highlight a few new avenues of research opened up by the perspective of "affordance as GVF", including using GVFs for orchestrating complex behaviors.
References in corpus (19)
- Playing Atari with Deep Reinforcement Learning
- Benchmarking Model-Based Reinforcement Learning
- Variational Intrinsic Control
- Dyna-Style Planning with Linear Function Approximation and Prioritized Sweeping
- Variational Option Discovery Algorithms
- Mapless Navigation among Dynamics with Social-safety-awareness: a reinforcement learning approach from 2D laser scans
- Spatial Action Maps for Mobile Manipulation
- Independently Controllable Factors
- Disentangling the independently controllable factors of variation by interacting with the world
- Visual Affordance and Function Understanding: A Survey
- What can I do here? A Theory of Affordances in Reinforcement Learning
- General Value Function Networks
- The Termination Critic
- Learning a world model and planning with a self-organizing, dynamic neural system
- Gradient Temporal-Difference Learning with Regularized Corrections
- Learning predictive representations in autonomous driving to improve deep reinforcement learning
- Learned human-agent decision-making, communication and joint action in a virtual reality environment
- LISPR: An Options Framework for Policy Reuse with Reinforcement Learning
- Off-Policy General Value Functions to Represent Dynamic Role Assignments in RoboCup 3D Soccer Simulation