Deep Deformable 3D Caricatures with Learned Shape Control

Abstract

A 3D caricature is an exaggerated 3D depiction of a human face. The goal of this paper is to model the variations of 3D caricatures in a compact parameter space so that we can provide a useful data-driven toolkit for handling 3D caricature deformations. To achieve the goal, we propose an MLP-based framework for building a deformable surface model, which takes a latent code and produces a 3D surface. In the framework, a SIREN MLP models a function that takes a 3D position on a fixed template surface and returns a 3D displacement vector for the input position. We create variations of 3D surfaces by learning a hypernetwork that takes a latent code and produces the parameters of the MLP. Once learned, our deformable model provides a nice editing space for 3D caricatures, supporting label-based semantic editing and point-handle-based deformation, both of which produce highly exaggerated and natural 3D caricature shapes. We also demonstrate other applications of our deformable model, such as automatic 3D caricature creation.

ACM SIGGRAPH 2022. For the project page, see https://ycjungsubhuman.github.io/DeepDeformable3DCaricatures

References in corpus (5)

• Implicit Neural Representations with Periodic Activation Functions
• DeepSketch2Face: A Deep Learning Based Sketching System for 3D Face and Caricature Modeling
• StyleCariGAN: Caricature Generation via StyleGAN Feature Map Modulation
• MeshGAN: Non-linear 3D Morphable Models of Faces
• 3D Magic Mirror: Automatic Video to 3D Caricature Translation