Rapid trial-and-error learning with simulation supports flexible tool use and physical reasoning

Abstract

Many animals, and an increasing number of artificial agents, display sophisticated capabilities to perceive and manipulate objects. But human beings remain distinctive in their capacity for flexible, creative tool use -- using objects in new ways to act on the world, achieve a goal, or solve a problem. To study this type of general physical problem solving, we introduce the Virtual Tools game. In this game, people solve a large range of challenging physical puzzles in just a handful of attempts. We propose that the flexibility of human physical problem solving rests on an ability to imagine the effects of hypothesized actions, while the efficiency of human search arises from rich action priors which are updated via observations of the world. We instantiate these components in the "Sample, Simulate, Update" (SSUP) model and show that it captures human performance across 30 levels of the Virtual Tools game. More broadly, this model provides a mechanism for explaining how people condense general physical knowledge into actionable, task-specific plans to achieve flexible and efficient physical problem-solving.

This manuscript is in press at PNAS. It is an extended version of a paper "Rapid Trial-and-Error Learning in Physical Problem Solving" accepted for oral presentation at the 41st Annual Meeting of the Cognitive Science Society (2019). It represents ongoing work on the part of the authors

References in corpus (9)

• Emergent Tool Use From Multi-Agent Autocurricula
• Efficient Off-Policy Meta-Reinforcement Learning via Probabilistic Context Variables
• Quantifying Generalization in Reinforcement Learning
• A Compositional Object-Based Approach to Learning Physical Dynamics
• A Review of Robot Learning for Manipulation: Challenges, Representations, and Algorithms
• Reasoning About Physical Interactions with Object-Oriented Prediction and Planning
• Switching Linear Dynamics for Variational Bayes Filtering
• Time-Agnostic Prediction: Predicting Predictable Video Frames
• Reasoning and Generalization in RL: A Tool Use Perspective

Cited by in corpus (7)

• Emergent Tool Use From Multi-Agent Autocurricula
• Bongard-LOGO: A New Benchmark for Human-Level Concept Learning and Reasoning
• A Review of Emerging Research Directions in Abstract Visual Reasoning
• Multi-Task Reinforcement Learning with Context-based Representations
• Physion: Evaluating Physical Prediction from Vision in Humans and Machines
• ToolNet: Using Commonsense Generalization for Predicting Tool Use for Robot Plan Synthesis
• Solving Physics Puzzles by Reasoning about Paths