Generating Emotive Gaits for Virtual Agents Using Affect-Based Autoregression

Abstract

We present a novel autoregression network to generate virtual agents that convey various emotions through their walking styles or gaits. Given the 3D pose sequences of a gait, our network extracts pertinent movement features and affective features from the gait. We use these features to synthesize subsequent gaits such that the virtual agents can express and transition between emotions represented as combinations of happy, sad, angry, and neutral. We incorporate multiple regularizations in the training of our network to simultaneously enforce plausible movements and noticeable emotions on the virtual agents. We also integrate our approach with an AR environment using a Microsoft HoloLens and can generate emotive gaits at interactive rates to increase the social presence. We evaluate how human observers perceive both the naturalness and the emotions from the generated gaits of the virtual agents in a web-based study. Our results indicate around 89% of the users found the naturalness of the gaits satisfactory on a five-point Likert scale, and the emotions they perceived from the virtual agents are statistically similar to the intended emotions of the virtual agents. We also use our network to augment existing gait datasets with emotive gaits and will release this augmented dataset for future research in emotion prediction and emotive gait synthesis. Our project website is available at https://gamma.umd.edu/gen_emotive_gaits/.

12 pages, 9 figures, 3 tables

References in corpus (12)

• Adam: A Method for Stochastic Optimization
• A Critical Review of Recurrent Neural Networks for Sequence Learning
• Scheduled Sampling for Sequence Prediction with Recurrent Neural Networks
• DeepMimic: Example-Guided Deep Reinforcement Learning of Physics-Based Character Skills
• Context Based Emotion Recognition using EMOTIC Dataset
• QuaterNet: A Quaternion-based Recurrent Model for Human Motion
• Unpaired Motion Style Transfer from Video to Animation
• STEP: Spatial Temporal Graph Convolutional Networks for Emotion Perception from Gaits
• Identifying Emotions from Walking using Affective and Deep Features
• Take an Emotion Walk: Perceiving Emotions from Gaits Using Hierarchical Attention Pooling and Affective Mapping
• Toward Intelligent Biped-Humanoids Gaits Generation
• The Liar's Walk: Detecting Deception with Gait and Gesture

Cited by in corpus (2)

• Text2Gestures: A Transformer-Based Network for Generating Emotive Body Gestures for Virtual Agents
• Speech2AffectiveGestures: Synthesizing Co-Speech Gestures with Generative Adversarial Affective Expression Learning