Long-term Human Motion Prediction with Scene Context
arXiv:2007.03672
Abstract
Human movement is goal-directed and influenced by the spatial layout of the objects in the scene. To plan future human motion, it is crucial to perceive the environment -- imagine how hard it is to navigate a new room with lights off. Existing works on predicting human motion do not pay attention to the scene context and thus struggle in long-term prediction. In this work, we propose a novel three-stage framework that exploits scene context to tackle this task. Given a single scene image and 2D pose histories, our method first samples multiple human motion goals, then plans 3D human paths towards each goal, and finally predicts 3D human pose sequences following each path. For stable training and rigorous evaluation, we contribute a diverse synthetic dataset with clean annotations. In both synthetic and real datasets, our method shows consistent quantitative and qualitative improvements over existing methods.
ECCV 2020 Oral. Dataset & Code: https://github.com/ZheC/GTA-IM-Dataset Video: https://people.eecs.berkeley.edu/~zhecao/hmp/index.html
References in corpus (11)
- Adam: A Method for Stochastic Optimization
- Auto-Encoding Variational Bayes
- Social Force Model for Pedestrian Dynamics
- Overcoming Limitations of Mixture Density Networks: A Sampling and Fitting Framework for Multimodal Future Prediction
- CornerNet-Lite: Efficient Keypoint Based Object Detection
- One-Shot Imitation from Observing Humans via Domain-Adaptive Meta-Learning
- Context-Aware Synthesis and Placement of Object Instances
- Geometric Pose Affordance: 3D Human Pose with Scene Constraints
- Human Motion Prediction via Spatio-Temporal Inpainting
- Auto-Conditioned Recurrent Networks for Extended Complex Human Motion Synthesis
- Predicting Camera Viewpoint Improves Cross-dataset Generalization for 3D Human Pose Estimation