Real-time Facial Surface Geometry from Monocular Video on Mobile GPUs

Abstract

We present an end-to-end neural network-based model for inferring an approximate 3D mesh representation of a human face from single camera input for AR applications. The relatively dense mesh model of 468 vertices is well-suited for face-based AR effects. The proposed model demonstrates super-realtime inference speed on mobile GPUs (100-1000+ FPS, depending on the device and model variant) and a high prediction quality that is comparable to the variance in manual annotations of the same image.

4 pages, 4 figures; CVPR Workshop on Computer Vision for Augmented and Virtual Reality, Long Beach, CA, USA, 2019

Cited by in corpus (7)

• MediaPipe Hands: On-device Real-time Hand Tracking
• BlazePose: On-device Real-time Body Pose tracking
• Attention Mesh: High-fidelity Face Mesh Prediction in Real-time
• GANonymization: A GAN-based Face Anonymization Framework for Preserving Emotional Expressions
• Real-time Pupil Tracking from Monocular Video for Digital Puppetry
• Portrait Shadow Manipulation
• Real-time RGBD-based Extended Body Pose Estimation