Predicting the Physical Dynamics of Unseen 3D Objects
arXiv:2001.06291
Abstract
Machines that can predict the effect of physical interactions on the dynamics of previously unseen object instances are important for creating better robots and interactive virtual worlds. In this work, we focus on predicting the dynamics of 3D objects on a plane that have just been subjected to an impulsive force. In particular, we predict the changes in state - 3D position, rotation, velocities, and stability. Different from previous work, our approach can generalize dynamics predictions to object shapes and initial conditions that were unseen during training. Our method takes the 3D object's shape as a point cloud and its initial linear and angular velocities as input. We extract shape features and use a recurrent neural network to predict the full change in state at each time step. Our model can support training with data from both a physics engine or the real world. Experiments show that we can accurately predict the changes in state for unseen object geometries and initial conditions.
In Proceedings of Winter Conference on Applications of Computer Vision (WACV) 2020. arXiv admin note: text overlap with arXiv:1901.00466
References in corpus (11)
- A Compositional Object-Based Approach to Learning Physical Dynamics
- Learning Physical Intuition of Block Towers by Example
- Visual Interaction Networks
- To Fall Or Not To Fall: A Visual Approach to Physical Stability Prediction
- Unsupervised Learning of Latent Physical Properties Using Perception-Prediction Networks
- Learning A Physical Long-term Predictor
- IntPhys: A Framework and Benchmark for Visual Intuitive Physics Reasoning
- SE3-Pose-Nets: Structured Deep Dynamics Models for Visuomotor Planning and Control
- SMASH: Physics-guided Reconstruction of Collisions from Videos
- Learning to Represent Mechanics via Long-term Extrapolation and Interpolation
- Learning Generalizable Physical Dynamics of 3D Rigid Objects