9 papers
Towards Flexible, Natural, Efficient Interaction for Conversational Talking Face Generation
Baiqin Wang, Sen Chen, Jiankuo Zhao +3
Conversational talking face generation has recently attracted increasing attention, aiming to synthesize interactive talking videos where characters speak, listen, and respond dyna…
Rosetta: Composable Native Multimodal Pretraining
Xiangyue Liu, Zijian Zhang, Miles Yang +3
Achieving true artificial general intelligence requires foundation models capable of integrating new modalities without forgetting prior knowledge. However, accommodating continuou…
TurboTalk: Progressive Distillation for One-Step Audio-Driven Talking Avatar Generation
Xiangyu Liu, Feng Gao, Xiaomei Zhang +4
Existing audio-driven video digital human generation models rely on multi-step denoising, resulting in substantial computational overhead that severely limits their deployment in r…
Symbiotic-MoE: Unlocking the Synergy between Generation and Understanding
Xiangyue Liu, Zijian Zhang, Miles Yang +3
Empowering Large Multimodal Models (LMMs) with image generation often leads to catastrophic forgetting in understanding tasks due to severe gradient conflicts. While existing parad…
InfiniteTalk: Audio-driven Video Generation for Sparse-Frame Video Dubbing
Shaoshu Yang, Zhe Kong, Feng Gao +8
Recent breakthroughs in video AIGC have ushered in a transformative era for audio-driven human animation. However, conventional video dubbing techniques remain constrained to mouth…
NoiseAR: AutoRegressing Initial Noise Prior for Diffusion Models
Zeming Li, Xiangyue Liu, Xiangyu Zhang +2
Diffusion models have emerged as powerful generative frameworks, creating data samples by progressively denoising an initial random state. Traditionally, this initial state is samp…
GaussianAvatar-Editor: Photorealistic Animatable Gaussian Head Avatar Editor
Xiangyue Liu, Kunming Luo, Heng Li +4
We introduce GaussianAvatar-Editor, an innovative framework for text-driven editing of animatable Gaussian head avatars that can be fully controlled in expression, pose, and viewpo…
MVBoost: Boost 3D Reconstruction with Multi-View Refinement
Xiangyu Liu, Xiaomei Zhang, Zhiyuan Ma +2
Recent advancements in 3D object reconstruction have been remarkable, yet most current 3D models rely heavily on existing 3D datasets. The scarcity of diverse 3D datasets results i…
GenN2N: Generative NeRF2NeRF Translation
Xiangyue Liu, Han Xue, Kunming Luo +2
We present GenN2N, a unified NeRF-to-NeRF translation framework for various NeRF translation tasks such as text-driven NeRF editing, colorization, super-resolution, inpainting, etc…