3D Gaussian Blendshapes for Head Avatar Animation

Abstract

We introduce 3D Gaussian blendshapes for modeling photorealistic head avatars. Taking a monocular video as input, we learn a base head model of neutral expression, along with a group of expression blendshapes, each of which corresponds to a basis expression in classical parametric face models. Both the neutral model and expression blendshapes are represented as 3D Gaussians, which contain a few properties to depict the avatar appearance. The avatar model of an arbitrary expression can be effectively generated by combining the neutral model and expression blendshapes through linear blending of Gaussians with the expression coefficients. High-fidelity head avatar animations can be synthesized in real time using Gaussian splatting. Compared to state-of-the-art methods, our Gaussian blendshape representation better captures high-frequency details exhibited in input video, and achieves superior rendering performance.

arXiv commentACM SIGGRAPH Conference Proceedings 2024

References in corpus (3)

• Instant Neural Graphics Primitives with a Multiresolution Hash Encoding
• Reconstructing Personalized Semantic Facial NeRF Models From Monocular Video
• AvatarMAV: Fast 3D Head Avatar Reconstruction Using Motion-Aware Neural Voxels

Cited by in corpus (5)

• High-quality Animatable Eyelid Shapes from Lightweight Captures
• SOAP: Style-Omniscient Animatable Portraits
• SketchRodGS: Sketch-based Extraction of Slender Geometries for Animating Gaussian Splatting Scenes
• Advancing Talking Head Generation: A Comprehensive Survey of Multi-Modal Methodologies, Datasets, Evaluation Metrics, and Loss Functions
• LoDAvatar: Hierarchical Embedding and Selective Detail Enhancement for Adaptive Levels of Detail Gaussian Avatars