1.2k citations · 1.7k across the 48 of their papers we have counts for
54 papers
STEER: Steerable Dyadic Head Avatars
Kartik Teotia, Helge Rhodin, Hyeongwoo Kim +2
Facial movement and expression are central to face-to-face communication, conveying turn-taking, attention, agreement, and engagement alongside speech. While speech-driven facial a…
E-3DPSM: A State Machine for Event-Based Egocentric 3D Human Pose Estimation
Mayur Deshmukh, Hiroyasu Akada, Helge Rhodin +2
Event cameras offer multiple advantages in monocular egocentric 3D human pose estimation from head-mounted devices, such as millisecond temporal resolution, high dynamic range, and…
Audio-Driven Universal Gaussian Head Avatars
Kartik Teotia, Helge Rhodin, Mohit Mendiratta +3
We introduce the first method for audio-driven universal photorealistic avatar synthesis, combining a person-agnostic speech model with our novel Universal Head Avatar Prior (UHAP)…
Follow My Hold: Hand-Object Interaction Reconstruction through Geometric Guidance
Ayce Idil Aytekin, Helge Rhodin, Rishabh Dabral +1
We propose a novel diffusion-based framework for reconstructing 3D geometry of hand-held objects from monocular RGB images by leveraging hand-object interaction as geometric guidan…
DreamTexture: Shape from Virtual Texture with Analysis by Augmentation
Ananta R. Bhattarai, Xingzhe He, Alla Sheffer +1
DreamFusion established a new paradigm for unsupervised 3D reconstruction from virtual views by combining advances in generative models and differentiable rendering. However, the u…
Gaussian Shadow Casting for Neural Characters
Luis Bolanos, Shih-Yang Su, Helge Rhodin
Neural character models can now reconstruct detailed geometry and texture from video, but they lack explicit shadows and shading, leading to artifacts when generating novel views a…
A Data Perspective on Enhanced Identity Preservation for Diffusion Personalization
Xingzhe He, Zhiwen Cao, Nicholas Kolkin +4
Large text-to-image models have revolutionized the ability to generate imagery using natural language. However, particularly unique or personal visual concepts, such as pets and fu…
Pose Modulated Avatars from Video
Chunjin Song, Bastian Wandt, Helge Rhodin
It is now possible to reconstruct dynamic human motion and shape from a sparse set of cameras using Neural Radiance Fields (NeRF) driven by an underlying skeleton. However, a chall…
Mirror-Aware Neural Humans
Daniel Ajisafe, James Tang, Shih-Yang Su +2
Human motion capture either requires multi-camera systems or is unreliable when using single-view input due to depth ambiguities. Meanwhile, mirrors are readily available in urban…
NPC: Neural Point Characters from Video
Shih-Yang Su, Timur Bagautdinov, Helge Rhodin
High-fidelity human 3D models can now be learned directly from videos, typically by combining a template-based surface model with neural representations. However, obtaining a templ…
Few-shot Geometry-Aware Keypoint Localization
Xingzhe He, Gaurav Bharaj, David Ferman +2
Supervised keypoint localization methods rely on large manually labeled image datasets, where objects can deform, articulate, or occlude. However, creating such large keypoint labe…
Scaling Neural Face Synthesis to High FPS and Low Latency by Neural Caching
Frank Yu, Sid Fels, Helge Rhodin
Recent neural rendering approaches greatly improve image quality, reaching near photorealism. However, the underlying neural networks have high runtime, precluding telepresence and…
UNeRF: Time and Memory Conscious U-Shaped Network for Training Neural Radiance Fields
Abiramy Kuganesan, Shih-yang Su, James J. Little +1
Neural Radiance Fields (NeRFs) increase reconstruction detail for novel view synthesis and scene reconstruction, with applications ranging from large static scenes to dynamic human…
LatentKeypointGAN: Controlling Images via Latent Keypoints -- Extended Abstract
Xingzhe He, Bastian Wandt, Helge Rhodin
Generative adversarial networks (GANs) can now generate photo-realistic images. However, how to best control the image content remains an open challenge. We introduce LatentKeypoin…
AutoLink: Self-supervised Learning of Human Skeletons and Object Outlines by Linking Keypoints
Xingzhe He, Bastian Wandt, Helge Rhodin
Structured representations such as keypoints are widely used in pose transfer, conditional image generation, animation, and 3D reconstruction. However, their supervised learning re…
DANBO: Disentangled Articulated Neural Body Representations via Graph Neural Networks
Shih-Yang Su, Timur Bagautdinov, Helge Rhodin
Deep learning greatly improved the realism of animatable human models by learning geometry and appearance from collections of 3D scans, template meshes, and multi-view imagery. Hig…
A Simple Method to Boost Human Pose Estimation Accuracy by Correcting the Joint Regressor for the Human3.6m Dataset
Eric Hedlin, Helge Rhodin, Kwang Moo Yi
Many human pose estimation methods estimate Skinned Multi-Person Linear (SMPL) models and regress the human joints from these SMPL estimates. In this work, we show that the most wi…
Domain Knowledge-Informed Self-Supervised Representations for Workout Form Assessment
Paritosh Parmar, Amol Gharat, Helge Rhodin
Maintaining proper form while exercising is important for preventing injuries and maximizing muscle mass gains. Detecting errors in workout form naturally requires estimating human…
Improved 2D Keypoint Detection in Out-of-Balance and Fall Situations -- combining input rotations and a kinematic model
Michael Zwölfer, Dieter Heinrich, Kurt Schindelwig +4
Injury analysis may be one of the most beneficial applications of deep learning based human pose estimation. To facilitate further research on this topic, we provide an injury spec…
ElePose: Unsupervised 3D Human Pose Estimation by Predicting Camera Elevation and Learning Normalizing Flows on 2D Poses
Bastian Wandt, James J. Little, Helge Rhodin
Human pose estimation from single images is a challenging problem that is typically solved by supervised learning. Unfortunately, labeled training data does not yet exist for many…
AdaptPose: Cross-Dataset Adaptation for 3D Human Pose Estimation by Learnable Motion Generation
Mohsen Gholami, Bastian Wandt, Helge Rhodin +2
This paper addresses the problem of cross-dataset generalization of 3D human pose estimation models. Testing a pre-trained 3D pose estimator on a new dataset results in a major per…
GANSeg: Learning to Segment by Unsupervised Hierarchical Image Generation
Xingzhe He, Bastian Wandt, Helge Rhodin
Segmenting an image into its parts is a frequent preprocess for high-level vision tasks such as image editing. However, annotating masks for supervised training is expensive. Weakl…
NRST: Non-rigid Surface Tracking from Monocular Video
Marc Habermann, Weipeng Xu, Helge Rhodin +3
We propose an efficient method for non-rigid surface tracking from monocular RGB videos. Given a video and a template mesh, our algorithm sequentially registers the template non-ri…
TriPose: A Weakly-Supervised 3D Human Pose Estimation via Triangulation from Video
Mohsen Gholami, Ahmad Rezaei, Helge Rhodin +2
Estimating 3D human poses from video is a challenging problem. The lack of 3D human pose annotations is a major obstacle for supervised training and for generalization to unseen da…