MoGlow: Probabilistic and controllable motion synthesis using normalising flows

Abstract

Data-driven modelling and synthesis of motion is an active research area with applications that include animation, games, and social robotics. This paper introduces a new class of probabilistic, generative, and controllable motion-data models based on normalising flows. Models of this kind can describe highly complex distributions, yet can be trained efficiently using exact maximum likelihood, unlike GANs or VAEs. Our proposed model is autoregressive and uses LSTMs to enable arbitrarily long time-dependencies. Importantly, is is also causal, meaning that each pose in the output sequence is generated without access to poses or control inputs from future time steps; this absence of algorithmic latency is important for interactive applications with real-time motion control. The approach can in principle be applied to any type of motion since it does not make restrictive, task-specific assumptions regarding the motion or the character morphology. We evaluate the models on motion-capture datasets of human and quadruped locomotion. Objective and subjective results show that randomly-sampled motion from the proposed method outperforms task-agnostic baselines and attains a motion quality close to recorded motion capture.

arXiv comment14 pages, 5 figures, published in ACM Transactions on Graphics and presented at SIGGRAPH Asia 2020

References in corpus (5)

• Efficiently Trainable Text-to-Speech System Based on Deep Convolutional Networks with Guided Attention
• Character Controllers Using Motion VAEs
• Robust Motion In-betweening
• Analyzing Input and Output Representations for Speech-Driven Gesture Generation
• Speech-driven Animation with Meaningful Behaviors

Cited by in corpus (27)

• Rhythmic Gesticulator: Rhythm-Aware Co-Speech Gesture Synthesis with Hierarchical Neural Embeddings
• A Comprehensive Review of Data-Driven Co-Speech Gesture Generation
• ControlVAE: Model-Based Learning of Generative Controllers for Physics-Based Characters
• GANimator: Neural Motion Synthesis from a Single Sequence
• Transflower: probabilistic autoregressive dance generation with multimodal attention
• A Survey on Deep Learning for Skeleton-Based Human Animation
• Rhythm is a Dancer: Music-Driven Motion Synthesis with Global Structure
• Let's Face It: Probabilistic Multi-modal Interlocutor-aware Generation of Facial Gestures in Dyadic Settings
• Passing a Non-verbal Turing Test: Evaluating Gesture Animations Generated from Speech
• Conditional Motion In-betweening
• Speech2Properties2Gestures: Gesture-Property Prediction as a Tool for Generating Representational Gestures from Speech
• Generating coherent spontaneous speech and gesture from text
• BodyFormer: Semantics-guided 3D Body Gesture Synthesis with Transformer
• Learning to Generate Pointing Gestures in Situated Embodied Conversational Agents
• Integrated Speech and Gesture Synthesis
• GesGPT: Speech Gesture Synthesis With Text Parsing from ChatGPT
• Pose Representations for Deep Skeletal Animation
• MOVIN: Real-time Motion Capture using a Single LiDAR
• A causal convolutional neural network for multi-subject motion modeling and generation
• ReNeLiB: Real-time Neural Listening Behavior Generation for Socially Interactive Agents
• A Two-part Transformer Network for Controllable Motion Synthesis
• Improving Sequential Latent Variable Models with Autoregressive Flows
• StackFLOW: Monocular Human-Object Reconstruction by Stacked Normalizing Flow with Offset
• ELMO: Enhanced Real-time LiDAR Motion Capture through Upsampling
• Real-time Diverse Motion In-betweening with Space-time Control
• Multi-Resolution Generative Modeling of Human Motion from Limited Data
• PFPN: Continuous Control of Physically Simulated Characters using Particle Filtering Policy Network