5 papers
UniMo: Unifying 2D Video and 3D Human Motion with an Autoregressive Framework
Youxin Pang, Yong Zhang, Ruizhi Shao +5
We propose UniMo, an innovative autoregressive model for joint modeling of 2D human videos and 3D human motions within a unified framework, enabling simultaneous generation and und…
DevilSight: Augmenting Monocular Human Avatar Reconstruction through a Virtual Perspective
Yushuo Chen, Ruizhi Shao, Youxin Pang +4
We present a novel framework to reconstruct human avatars from monocular videos. Recent approaches have struggled either to capture the fine-grained dynamic details from the input…
SViMo: Synchronized Diffusion for Video and Motion Generation in Hand-object Interaction Scenarios
Lingwei Dang, Ruizhi Shao, Hongwen Zhang +3
Hand-Object Interaction (HOI) generation has significant application potential. However, current 3D HOI motion generation approaches heavily rely on predefined 3D object models and…
Interspatial Attention for Efficient 4D Human Video Generation
Ruizhi Shao, Yinghao Xu, Yujun Shen +5
Generating photorealistic videos of digital humans in a controllable manner is crucial for a plethora of applications. Existing approaches either build on methods that employ templ…
The Language of Motion: Unifying Verbal and Non-verbal Language of 3D Human Motion
Changan Chen, Juze Zhang, Shrinidhi K. Lakshmikanth +5
Human communication is inherently multimodal, involving a combination of verbal and non-verbal cues such as speech, facial expressions, and body gestures. Modeling these behaviors…