Dynamic Neural Garments

Abstract

A vital task of the wider digital human effort is the creation of realistic garments on digital avatars, both in the form of characteristic fold patterns and wrinkles in static frames as well as richness of garment dynamics under avatars' motion. Existing workflow of modeling, simulation, and rendering closely replicates the physics behind real garments, but is tedious and requires repeating most of the workflow under changes to characters' motion, camera angle, or garment resizing. Although data-driven solutions exist, they either focus on static scenarios or only handle dynamics of tight garments. We present a solution that, at test time, takes in body joint motion to directly produce realistic dynamic garment image sequences. Specifically, given the target joint motion sequence of an avatar, we propose dynamic neural garments to jointly simulate and render plausible dynamic garment appearance from an unseen viewpoint. Technically, our solution generates a coarse garment proxy sequence, learns deep dynamic features attached to this template, and neurally renders the features to produce appearance changes such as folds, wrinkles, and silhouettes. We demonstrate generalization behavior to both unseen motion and unseen camera views. Further, our network can be fine-tuned to adopt to new body shape and/or background images. We also provide comparisons against existing neural rendering and image sequence translation approaches, and report clear quantitative improvements.

13 pages

References in corpus (7)

• GANs Trained by a Two Time-Scale Update Rule Converge to a Local Nash Equilibrium
• NeRF: Representing Scenes as Neural Radiance Fields for View Synthesis
• First Order Motion Model for Image Animation
• Neural Scene Flow Fields for Space-Time View Synthesis of Dynamic Scenes
• A Pixel-Based Framework for Data-Driven Clothing
• Learning Compositional Radiance Fields of Dynamic Human Heads
• SSN: Soft Shadow Network for Image Compositing