7 papers
How to Build Digital Humans? From Priors to Photorealistic Avatars
Wojciech Zielonka, Tobias Kirschstein, Timo Bolkart +8
This state-of-the-art report provides an overview of controllable 3D human avatar creation. We describe current 3D avatar systems, which typically consist of three stages: (i) lear…
GeoDiff4D: Geometry-Aware Diffusion for 4D Head Avatar Reconstruction
Chao Xu, Xiaochen Zhao, Xiang Deng +4
Reconstructing photorealistic and animatable 4D head avatars from a single portrait image remains a fundamental challenge in computer vision. While diffusion models have enabled re…
MAViD: A Multimodal Framework for Audio-Visual Dialogue Understanding and Generation
Youxin Pang, Jiajun Liu, Lingfeng Tan +6
We propose MAViD, a novel Multimodal framework for Audio-Visual Dialogue understanding and generation. Existing approaches primarily focus on non-interactive systems and are limite…
U-Mind: A Unified Framework for Real-Time Multimodal Interaction with Audiovisual Generation
Xiang Deng, Feng Gao, Yong Zhang +5
Full-stack multimodal interaction in real-time is a central goal in building intelligent embodied agents capable of natural, dynamic communication. However, existing systems are ei…
UniMo: Unifying 2D Video and 3D Human Motion with an Autoregressive Framework
Youxin Pang, Yong Zhang, Ruizhi Shao +5
We propose UniMo, an innovative autoregressive model for joint modeling of 2D human videos and 3D human motions within a unified framework, enabling simultaneous generation and und…
GMTalker: Gaussian Mixture-based Audio-Driven Emotional Talking Video Portraits
Yibo Xia, Lizhen Wang, Xiang Deng +3
Synthesizing high-fidelity and emotion-controllable talking video portraits, with audio-lip sync, vivid expressions, realistic head poses, and eye blinks, has been an important and…