33 citations · 51 across the 26 of their papers we have counts for
35 papers
A Benchmark for Spatially Grounded Gesture Generation
Anna Deichler, Rishabh Dabral, Fethiye Irmak Dogan +2
Communication in shared space interweaves verbal and non-verbal signals, and pointing gestures anchor language to the environment: "put the cup on that one" is uninterpretable with…
Towards Real-World Wearable Motion Reconstruction
Andrea Boscolo Camiletto, Rishabh Dabral, Eduardo Alvarado +3
The modern-day surge in popularity of wearable devices poses a fundamentally unique motion capture problem: reconstructing full-body movement from any set of sensing hardware worn…
SceMoS: Scene-Aware 3D Human Motion Synthesis by Planning with Geometry-Grounded Tokens
Anindita Ghosh, Vladislav Golyanik, Taku Komura +3
Synthesizing text-driven 3D human motion within realistic scenes requires learning both semantic intent ("walk to the couch") and physical feasibility (e.g., avoiding collisions).…
VHOI: Controllable Video Generation of Human-Object Interactions from Sparse Trajectories via Motion Densification
Wanyue Zhang, Lin Geng Foo, Thabo Beeler +2
Synthesizing realistic human-object interactions (HOI) in video is challenging due to the complex, instance-specific interaction dynamics of both humans and objects. Incorporating…
Relightable Holoported Characters: Capturing and Relighting Dynamic Human Performance from Sparse Views
Kunwar Maheep Singh, Jianchun Chen, Vladislav Golyanik +5
We present Relightable Holoported Characters (RHC), a novel person-specific method for free-view rendering and relighting of full-body and highly dynamic humans solely observed fro…
Towards Reliable Human Evaluations in Gesture Generation: Insights from a Community-Driven State-of-the-Art Benchmark
Rajmund Nagy, Hendric Voss, Thanh Hoang-Minh +18
We review human evaluation practices in automatic, speech-driven 3D gesture generation and find a lack of standardisation and frequent use of flawed experimental setups. This leads…
PractiLight: Practical Light Control Using Foundational Diffusion Models
Yotam Erel, Rishabh Dabral, Vladislav Golyanik +2
Light control in generated images is a difficult task, posing specific challenges, spanning over the entire image and frequency spectrum. Most approaches tackle this problem by tra…
Follow My Hold: Hand-Object Interaction Reconstruction through Geometric Guidance
Ayce Idil Aytekin, Helge Rhodin, Rishabh Dabral +1
We propose a novel diffusion-based framework for reconstructing 3D geometry of hand-held objects from monocular RGB images by leveraging hand-object interaction as geometric guidan…
Physics-based Human Pose Estimation from a Single Moving RGB Camera
Ayce Idil Aytekin, Chuqiao Li, Diogo Luvizon +4
Most monocular and physics-based human pose tracking methods, while achieving state-of-the-art results, suffer from artifacts when the scene does not have a strictly flat ground pl…
Attention (as Discrete-Time Markov) Chains
Yotam Erel, Olaf Dünkel, Rishabh Dabral +3
We introduce a new interpretation of the attention matrix as a discrete-time Markov chain. Our interpretation sheds light on common operations involving attention scores such as se…
DuetGen: Music Driven Two-Person Dance Generation via Hierarchical Masked Modeling
Anindita Ghosh, Bing Zhou, Rishabh Dabral +5
We present DuetGen, a novel framework for generating interactive two-person dances from music. The key challenge of this task lies in the inherent complexities of two-person dance…
Ego4o: Egocentric Human Motion Capture and Understanding from Multi-Modal Input
Jian Wang, Rishabh Dabral, Diogo Luvizon +4
This work focuses on tracking and understanding human motion using consumer wearable devices, such as VR/AR headsets, smart glasses, cellphones, and smartwatches. These devices pro…
FRAME: Floor-aligned Representation for Avatar Motion from Egocentric Video
Andrea Boscolo Camiletto, Jian Wang, Eduardo Alvarado +4
Egocentric motion capture with a head-mounted body-facing stereo camera is crucial for VR and AR applications but presents significant challenges such as heavy occlusions and limit…
3HANDS Dataset: Learning from Humans for Generating Naturalistic Handovers with Supernumerary Robotic Limbs
Artin Saberpour Abadian, Yi-Chi Liao, Ata Otaran +5
Supernumerary robotic limbs (SRLs) are robotic structures integrated closely with the user's body, which augment human physical capabilities and necessitate seamless, naturalistic…
Betsu-Betsu: Multi-View Separable 3D Reconstruction of Two Interacting Objects
Suhas Gopal, Rishabh Dabral, Vladislav Golyanik +1
Separable 3D reconstruction of multiple objects from multi-view RGB images -- resulting in two different 3D shapes for the two objects with a clear separation between them -- remai…
Real-time Free-view Human Rendering from Sparse-view RGB Videos using Double Unprojected Textures
Guoxing Sun, Rishabh Dabral, Heming Zhu +3
Real-time free-view human rendering from sparse-view RGB inputs is a challenging task due to the sensor scarcity and the tight time budget. To ensure efficiency, recent methods lev…
Retrieving Semantics from the Deep: an RAG Solution for Gesture Synthesis
M. Hamza Mughal, Rishabh Dabral, Merel C. J. Scholman +2
Non-verbal communication often comprises of semantically rich gestures that help convey the meaning of an utterance. Producing such semantic co-speech gestures has been a major cha…
BimArt: A Unified Approach for the Synthesis of 3D Bimanual Interaction with Articulated Objects
Wanyue Zhang, Rishabh Dabral, Vladislav Golyanik +5
We present BimArt, a novel generative approach for synthesizing 3D bimanual hand interactions with articulated objects. Unlike prior works, we do not rely on a reference grasp, a c…
PocoLoco: A Point Cloud Diffusion Model of Human Shape in Loose Clothing
Siddharth Seth, Rishabh Dabral, Diogo Luvizon +4
Modeling a human avatar that can plausibly deform to articulations is an active area of research. We present PocoLoco -- the first template-free, point-based, pose-conditioned gene…
ConvoFusion: Multi-Modal Conversational Diffusion for Co-Speech Gesture Synthesis
Muhammad Hamza Mughal, Rishabh Dabral, Ikhsanul Habibie +3
Gestures play a key role in human communication. Recent methods for co-speech gesture generation, while managing to generate beat-aligned motions, struggle generating gestures that…
MetaCap: Meta-learning Priors from Multi-View Imagery for Sparse-view Human Performance Capture and Rendering
Guoxing Sun, Rishabh Dabral, Pascal Fua +2
Faithful human performance capture and free-view rendering from sparse RGB observations is a long-standing problem in Vision and Graphics. The main challenges are the lack of obser…
3D-QAE: Fully Quantum Auto-Encoding of 3D Point Clouds
Lakshika Rathi, Edith Tretschk, Christian Theobalt +2
Existing methods for learning 3D representations are deep neural networks trained and tested on classical hardware. Quantum machine learning architectures, despite their theoretica…
ReMoS: 3D Motion-Conditioned Reaction Synthesis for Two-Person Interactions
Anindita Ghosh, Rishabh Dabral, Vladislav Golyanik +2
Current approaches for 3D human motion synthesis generate high quality animations of digital humans performing a wide variety of actions and gestures. However, a notable technologi…
ROAM: Robust and Object-Aware Motion Generation Using Neural Pose Descriptors
Wanyue Zhang, Rishabh Dabral, Thomas Leimkühler +3
Existing automatic approaches for 3D virtual character motion synthesis supporting scene interactions do not generalise well to new objects outside training distributions, even whe…