Diverse Human Motion Prediction Guided by Multi-Level Spatial-Temporal Anchors
arXiv:2302.04860 · doi:10.1007/978-3-031-20047-2_15
Abstract
Predicting diverse human motions given a sequence of historical poses has received increasing attention. Despite rapid progress, existing work captures the multi-modal nature of human motions primarily through likelihood-based sampling, where the mode collapse has been widely observed. In this paper, we propose a simple yet effective approach that disentangles randomly sampled codes with a deterministic learnable component named anchors to promote sample precision and diversity. Anchors are further factorized into spatial anchors and temporal anchors, which provide attractively interpretable control over spatial-temporal disparity. In principle, our spatial-temporal anchor-based sampling (STARS) can be applied to different motion predictors. Here we propose an interaction-enhanced spatial-temporal graph convolutional network (IE-STGCN) that encodes prior knowledge of human motions (e.g., spatial locality), and incorporate the anchors into it. Extensive experiments demonstrate that our approach outperforms state of the art in both stochastic and deterministic prediction, suggesting it as a unified framework for modeling human motions. Our code and pretrained models are available at https://github.com/Sirui-Xu/STARS.
arXiv commentECCV 2022 (Oral); Project Page: https://sirui-xu.github.io/STARS/
References in corpus (16)
- Adam: A Method for Stochastic Optimization
- Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift
- SSD: Single Shot MultiBox Detector
- PyTorch: An Imperative Style, High-Performance Deep Learning Library
- An Empirical Evaluation of Generic Convolutional and Recurrent Networks for Sequence Modeling
- On the difficulty of training Recurrent Neural Networks
- Spatio-Temporal Graph Convolutional Networks: A Deep Learning Framework for Traffic Forecasting
- Variational Inference with Normalizing Flows
- Deep Unsupervised Clustering with Gaussian Mixture Variational Autoencoders
- Character Controllers Using Motion VAEs
- Diverse Trajectory Forecasting with Determinantal Point Processes
- Human Motion Modeling using DVGANs
- Deep representation learning for human motion prediction and classification
- Residual Force Control for Agile Human Behavior Imitation and Extended Motion Synthesis
- Accurate and Diverse Sampling of Sequences based on a "Best of Many" Sample Objective
- DMS-GCN: Dynamic Mutiscale Spatiotemporal Graph Convolutional Networks for Human Motion Prediction