DLow: Diversifying Latent Flows for Diverse Human Motion Prediction

Abstract

Deep generative models are often used for human motion prediction as they are able to model multi-modal data distributions and characterize diverse human behavior. While much care has been taken into designing and learning deep generative models, how to efficiently produce diverse samples from a deep generative model after it has been trained is still an under-explored problem. To obtain samples from a pretrained generative model, most existing generative human motion prediction methods draw a set of independent Gaussian latent codes and convert them to motion samples. Clearly, this random sampling strategy is not guaranteed to produce diverse samples for two reasons: (1) The independent sampling cannot force the samples to be diverse; (2) The sampling is based solely on likelihood which may only produce samples that correspond to the major modes of the data distribution. To address these problems, we propose a novel sampling method, Diversifying Latent Flows (DLow), to produce a diverse set of samples from a pretrained deep generative model. Unlike random (independent) sampling, the proposed DLow sampling method samples a single random variable and then maps it with a set of learnable mapping functions to a set of correlated latent codes. The correlated latent codes are then decoded into a set of correlated samples. During training, DLow uses a diversity-promoting prior over samples as an objective to optimize the latent mappings to improve sample diversity. The design of the prior is highly flexible and can be customized to generate diverse motions with common features (e.g., similar leg motion but diverse upper-body motion). Our experiments demonstrate that DLow outperforms state-of-the-art baseline methods in terms of sample diversity and accuracy. Our code is released on the project page: https://www.ye-yuan.com/dlow.

arXiv commentECCV 2020. Project Page: https://www.ye-yuan.com/dlow

References in corpus (20)

• Adam: A Method for Stochastic Optimization
• Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modeling
• Deep Unsupervised Clustering with Gaussian Mixture Variational Autoencoders
• VEEGAN: Reducing Mode Collapse in GANs using Implicit Variational Learning
• Generalized Fisher Score for Feature Selection
• InfoVAE: Information Maximizing Variational Autoencoders
• Mode Regularized Generative Adversarial Networks
• QuaterNet: A Quaternion-based Recurrent Model for Human Motion
• Cyclical Annealing Schedule: A Simple Approach to Mitigating KL Vanishing
• Lagging Inference Networks and Posterior Collapse in Variational Autoencoders
• Diversity-Sensitive Conditional Generative Adversarial Networks
• Diverse Trajectory Forecasting with Determinantal Point Processes
• Human Motion Modeling using DVGANs
• Expectation-Maximization for Learning Determinantal Point Processes
• Residual Force Control for Agile Human Behavior Imitation and Extended Motion Synthesis
• Semi-Amortized Variational Autoencoders
• PTP: Parallelized Tracking and Prediction with Graph Neural Networks and Diversity Sampling
• Human Motion Prediction via Spatio-Temporal Inpainting
• GDPP: Learning Diverse Generations Using Determinantal Point Process
• Learning Variations in Human Motion via Mix-and-Match Perturbation

Cited by in corpus (2)

• Residual Force Control for Agile Human Behavior Imitation and Extended Motion Synthesis
• Generative Hybrid Representations for Activity Forecasting with No-Regret Learning