High-quality Animatable Eyelid Shapes from Lightweight Captures

Abstract

High-quality eyelid reconstruction and animation are challenging for the subtle details and complicated deformations. Previous works usually suffer from the trade-off between the capture costs and the quality of details. In this paper, we propose a novel method that can achieve detailed eyelid reconstruction and animation by only using an RGB video captured by a mobile phone. Our method utilizes both static and dynamic information of eyeballs (e.g., positions and rotations) to assist the eyelid reconstruction, cooperating with an automatic eyeball calibration method to get the required eyeball parameters. Furthermore, we develop a neural eyelid control module to achieve the semantic animation control of eyelids. To the best of our knowledge, we present the first method for high-quality eyelid reconstruction and animation from lightweight captures. Extensive experiments on both synthetic and real data show that our method can provide more detailed and realistic results compared with previous methods based on the same-level capture setups. The code is available at https://github.com/StoryMY/AniEyelid.

arXiv commentAccepted by SIGGRAPH Asia 2024

References in corpus (10)

• Instant Neural Graphics Primitives with a Multiresolution Hash Encoding
• 2D Gaussian Splatting for Geometrically Accurate Radiance Fields
• Fast Dynamic Radiance Fields with Time-Aware Neural Voxels
• Reconstructing Personalized Semantic Facial NeRF Models From Monocular Video
• AvatarMAV: Fast 3D Head Avatar Reconstruction Using Motion-Aware Neural Voxels
• 3D Gaussian Blendshapes for Head Avatar Animation
• FLARE: Fast Learning of Animatable and Relightable Mesh Avatars
• EyeNeRF: A Hybrid Representation for Photorealistic Synthesis, Animation and Relighting of Human Eyes
• Single-Shot Implicit Morphable Faces with Consistent Texture Parameterization
• Neural Parameterization for Dynamic Human Head Editing