1 citations · 2 across the 17 of their papers we have counts for
20 papers
HandsOnWorld: Unconstrained Egocentric Video Generation with Camera-Disentangled Hand Control
Yushuo Chen, Xiaoyu Shi, Xiaoshi Wu +3
We present HandsOnWorld, a framework for hand-controlled egocentric video generation that learns directly from unconstrained monocular video. Prior generators depend on 3D hand ann…
SynAgent: Generalizable Cooperative Humanoid Manipulation via Solo-to-Cooperative Agent Synergy
Wei Yao, Haohan Ma, Hongwen Zhang +5
Controllable cooperative humanoid manipulation is a fundamental yet challenging problem for embodied intelligence, due to severe data scarcity, complexities in multi-agent coordina…
X-NeMo: Expressive Neural Motion Reenactment via Disentangled Latent Attention
Xiaochen Zhao, Hongyi Xu, Guoxian Song +6
We propose X-NeMo, a novel zero-shot diffusion-based portrait animation pipeline that animates a static portrait using facial movements from a driving video of a different individu…
SViMo: Synchronized Diffusion for Video and Motion Generation in Hand-object Interaction Scenarios
Lingwei Dang, Ruizhi Shao, Hongwen Zhang +3
Hand-Object Interaction (HOI) generation has significant application potential. However, current 3D HOI motion generation approaches heavily rely on predefined 3D object models and…
GAF: Gaussian Action Field as a 4D Representation for Dynamic World Modeling in Robotic Manipulation
Ying Chai, Litao Deng, Ruizhi Shao +6
Accurate scene perception is critical for vision-based robotic manipulation. Existing approaches typically follow either a Vision-to-Action (V-A) paradigm, predicting actions direc…
HOSIG: Full-Body Human-Object-Scene Interaction Generation with Hierarchical Scene Perception
Wei Yao, Yunlian Sun, Hongwen Zhang +2
Generating high-fidelity full-body human interactions with dynamic objects and static scenes remains a critical challenge in computer graphics and animation. Existing methods for h…
ReCoM: Realistic Co-Speech Motion Generation with Recurrent Embedded Transformer
Yong Xie, Yunlian Sun, Hongwen Zhang +2
We present ReCoM, an efficient framework for generating high-fidelity and generalizable human body motions synchronized with speech. The core innovation lies in the Recurrent Embed…
A Plug-and-Play Physical Motion Restoration Approach for In-the-Wild High-Difficulty Motions
Youliang Zhang, Ronghui Li, Yachao Zhang +4
Extracting physically plausible 3D human motion from videos is a critical task. Although existing simulation-based motion imitation methods can enhance the physical quality of dail…
InterDance:Reactive 3D Dance Generation with Realistic Duet Interactions
Ronghui Li, Youliang Zhang, Yachao Zhang +6
Humans perform a variety of interactive motions, among which duet dance is one of the most challenging interactions. However, in terms of human motion generative models, existing w…
ManiVideo: Generating Hand-Object Manipulation Video with Dexterous and Generalizable Grasping
Youxin Pang, Ruizhi Shao, Jiajun Zhang +5
In this paper, we introduce ManiVideo, a novel method for generating consistent and temporally coherent bimanual hand-object manipulation videos from given motion sequences of hand…
Lodge++: High-quality and Long Dance Generation with Vivid Choreography Patterns
Ronghui Li, Hongwen Zhang, Yachao Zhang +6
We propose Lodge++, a choreography framework to generate high-quality, ultra-long, and vivid dances given the music and desired genre. To handle the challenges in computational eff…
ManiDext: Hand-Object Manipulation Synthesis via Continuous Correspondence Embeddings and Residual-Guided Diffusion
Jiajun Zhang, Yuxiang Zhang, Liang An +4
Dynamic and dexterous manipulation of objects presents a complex challenge, requiring the synchronization of hand motions with the trajectories of objects to achieve seamless and p…
HHMR: Holistic Hand Mesh Recovery by Enhancing the Multimodal Controllability of Graph Diffusion Models
Mengcheng Li, Hongwen Zhang, Yuxiang Zhang +3
Recent years have witnessed a trend of the deep integration of the generation and reconstruction paradigms. In this paper, we extend the ability of controllable generative models f…
OmniHands: Towards Robust 4D Hand Mesh Recovery via A Versatile Transformer
Dixuan Lin, Yuxiang Zhang, Mengcheng Li +5
In this paper, we introduce OmniHands, a universal approach to recovering interactive hand meshes and their relative movement from monocular or multi-view inputs. Our approach addr…
Lodge: A Coarse to Fine Diffusion Network for Long Dance Generation Guided by the Characteristic Dance Primitives
Ronghui Li, YuXiang Zhang, Yachao Zhang +5
We propose Lodge, a network capable of generating extremely long dance sequences conditioned on given music. We design Lodge as a two-stage coarse to fine diffusion architecture, a…
TACO: Benchmarking Generalizable Bimanual Tool-ACtion-Object Understanding
Yun Liu, Haolin Yang, Xu Si +5
Humans commonly work with multiple objects in daily life and can intuitively transfer manipulation skills to novel objects by understanding object functional regularities. However,…
Ins-HOI: Instance Aware Human-Object Interactions Recovery
Jiajun Zhang, Yuxiang Zhang, Hongwen Zhang +5
Accurately modeling detailed interactions between human/hand and object is an appealing yet challenging task. Current multi-view capture systems are only capable of reconstructing…
Layered 3D Human Generation via Semantic-Aware Diffusion Model
Yi Wang, Jian Ma, Ruizhi Shao +4
The generation of 3D clothed humans has attracted increasing attention in recent years. However, existing work cannot generate layered high-quality 3D humans with consistent body s…
SpeechAct: Towards Generating Whole-body Motion from Speech
Jinsong Zhang, Minjie Zhu, Yuxiang Zhang +2
This paper addresses the problem of generating whole-body motion from speech. Despite great successes, prior methods still struggle to produce reasonable and diverse whole-body mot…
ProxyCap: Real-time Monocular Full-body Capture in World Space via Human-Centric Proxy-to-Motion Learning
Yuxiang Zhang, Hongwen Zhang, Liangxiao Hu +4
Learning-based approaches to monocular motion capture have recently shown promising results by learning to regress in a data-driven manner. However, due to the challenges in data c…