9 papers
Beauty is in the AI of the beholder: MLLMs systematically overrate facial attractiveness
Santiago Grandas, Juan Sebastian Cely-Acosta, Mohit Mendiratta +2
Beauty assessments from Multimodal Large Language Models (MLLMs) are increasingly popular amongst users, companies, and aestheticians. This raises the question of whether these AI…
EmoteGPT: 3D Human Facial Expressions from Natural Language Descriptions
Haoran Wang, Mohit Mendiratta, Christian Theobalt +1
Precise control of 3D facial expressions from text is crucial for virtual avatars, animation, and human-computer interaction, yet existing text-to-3D methods jointly generate ident…
Audio-Driven Universal Gaussian Head Avatars
Kartik Teotia, Helge Rhodin, Mohit Mendiratta +3
We introduce the first method for audio-driven universal photorealistic avatar synthesis, combining a person-agnostic speech model with our novel Universal Head Avatar Prior (UHAP)…
GRMM: Real-Time High-Fidelity Gaussian Morphable Head Model with Learned Residuals
Mohit Mendiratta, Mayur Deshmukh, Kartik Teotia +3
3D Morphable Models (3DMMs) enable controllable facial geometry and expression editing for reconstruction, animation, and AR/VR, but traditional PCA-based mesh models are limited i…
TEDRA: Text-based Editing of Dynamic and Photoreal Actors
Basavaraj Sunagad, Heming Zhu, Mohit Mendiratta +3
Over the past years, significant progress has been made in creating photorealistic and drivable 3D avatars solely from videos of real humans. However, a core remaining challenge is…
FaceGPT: Self-supervised Learning to Chat about 3D Human Faces
Haoran Wang, Mohit Mendiratta, Christian Theobalt +1
We introduce FaceGPT, a self-supervised learning framework for Large Vision-Language Models (VLMs) to reason about 3D human faces from images and text. Typical 3D face reconstructi…
Zero-Shot Video Semantic Segmentation based on Pre-Trained Diffusion Models
Qian Wang, Abdelrahman Eldesokey, Mohit Mendiratta +4
We introduce the first zero-shot approach for Video Semantic Segmentation (VSS) based on pre-trained diffusion models. A growing research direction attempts to employ diffusion mod…
AvatarStudio: Text-driven Editing of 3D Dynamic Human Head Avatars
Mohit Mendiratta, Xingang Pan, Mohamed Elgharib +6
Capturing and editing full head performances enables the creation of virtual characters with various applications such as extended reality and media production. The past few years…
Egocentric Videoconferencing
Mohamed Elgharib, Mohit Mendiratta, Justus Thies +5
We introduce a method for egocentric videoconferencing that enables hands-free video calls, for instance by people wearing smart glasses or other mixed-reality devices. Videoconfer…