5 papers
MAViD: A Multimodal Framework for Audio-Visual Dialogue Understanding and Generation
Youxin Pang, Jiajun Liu, Lingfeng Tan +6
We propose MAViD, a novel Multimodal framework for Audio-Visual Dialogue understanding and generation. Existing approaches primarily focus on non-interactive systems and are limite…
U-Mind: A Unified Framework for Real-Time Multimodal Interaction with Audiovisual Generation
Xiang Deng, Feng Gao, Yong Zhang +5
Full-stack multimodal interaction in real-time is a central goal in building intelligent embodied agents capable of natural, dynamic communication. However, existing systems are ei…
UniMo: Unifying 2D Video and 3D Human Motion with an Autoregressive Framework
Youxin Pang, Yong Zhang, Ruizhi Shao +5
We propose UniMo, an innovative autoregressive model for joint modeling of 2D human videos and 3D human motions within a unified framework, enabling simultaneous generation and und…
DevilSight: Augmenting Monocular Human Avatar Reconstruction through a Virtual Perspective
Yushuo Chen, Ruizhi Shao, Youxin Pang +4
We present a novel framework to reconstruct human avatars from monocular videos. Recent approaches have struggled either to capture the fine-grained dynamic details from the input…
ManiVideo: Generating Hand-Object Manipulation Video with Dexterous and Generalizable Grasping
Youxin Pang, Ruizhi Shao, Jiajun Zhang +5
In this paper, we introduce ManiVideo, a novel method for generating consistent and temporally coherent bimanual hand-object manipulation videos from given motion sequences of hand…