To Fall Or Not To Fall: A Visual Approach to Physical Stability Prediction

Abstract

Understanding physical phenomena is a key competence that enables humans and animals to act and interact under uncertain perception in previously unseen environments containing novel object and their configurations. Developmental psychology has shown that such skills are acquired by infants from observations at a very early stage. In this paper, we contrast a more traditional approach of taking a model-based route with explicit 3D representations and physical simulation by an end-to-end approach that directly predicts stability and related quantities from appearance. We ask the question if and to what extent and quality such a skill can directly be acquired in a data-driven way bypassing the need for an explicit simulation. We present a learning-based approach based on simulated data that predicts stability of towers comprised of wooden blocks under different conditions and quantities related to the potential fall of the towers. The evaluation is carried out on synthetic data and compared to human judgments on the same stimuli.

References in corpus (2)

• Very Deep Convolutional Networks for Large-Scale Image Recognition
• Learning Visual Predictive Models of Physics for Playing Billiards

Cited by in corpus (27)

• Explainable Machine Learning for Scientific Insights and Discoveries
• Interaction Networks for Learning about Objects, Relations and Physics
• Flexible Neural Representation for Physics Prediction
• Learning Physical Intuition of Block Towers by Example
• Visual Interaction Networks
• Physics-Informed Supervised Residual Learning for Electromagnetic Modeling
• IntPhys: A Framework and Benchmark for Visual Intuitive Physics Reasoning
• O2O-Afford: Annotation-Free Large-Scale Object-Object Affordance Learning
• Physion: Evaluating Physical Prediction from Vision in Humans and Machines
• Answering Visual What-If Questions: From Actions to Predicted Scene Descriptions
• Object and Relation Centric Representations for Push Effect Prediction
• Human-like machine learning: limitations and suggestions
• Forward Prediction for Physical Reasoning
• Occlusion resistant learning of intuitive physics from videos
• Information-theoretic Model Identification and Policy Search using Physics Engines with Application to Robotic Manipulation
• Perceiving Physical Equation by Observing Visual Scenarios
• Learning Generalizable Physical Dynamics of 3D Rigid Objects
• ShapeStacks: Learning Vision-Based Physical Intuition for Generalised Object Stacking
• Acquiring Target Stacking Skills by Goal-Parameterized Deep Reinforcement Learning
• Go with the Flow: Perception-refined Physics Simulation
• Structured agents for physical construction
• Spatio-Temporal Image Boundary Extrapolation
• Interpretable Intuitive Physics Model
• Predicting the Physical Dynamics of Unseen 3D Objects
• PHYRE: A New Benchmark for Physical Reasoning
• Compositional Video Prediction
• Temporal and Physical Reasoning for Perception-Based Robotic Manipulation