11 papers
Talk Like You: Imitating How You Speak in Real-Time Talking Head Generation
Baiqin Wang, Zhixing Ding, Jijie Li +3
In daily life, each person exhibits unique speaking habits, leading to subtle yet consistent lip-shape variations even when pronouncing the same word. Although recent talking head…
Relightable 3D Avatar Reconstruction with Semantic-Adaptive Motion-Illumination Responses
Jiankuo Zhao, Xiangyu Zhu, Jijie Li +3
Reconstructing expressive and relightable 3D head avatars from monocular videos remains challenging in computer vision, as it requires accurate modeling of both non-rigid facial mo…
Object-Centric Conditioning for Visuomotor Flow Matching
Jijie Li, Xu Yang, Junhong Zou +4
Robot visuomotor policies are commonly formulated as autoregressive, diffusion-based, or more recently, flow matching models. Among them, Action-to-Action (A2A) flow matching impro…
Towards Flexible, Natural, Efficient Interaction for Conversational Talking Face Generation
Baiqin Wang, Sen Chen, Jiankuo Zhao +3
Conversational talking face generation has recently attracted increasing attention, aiming to synthesize interactive talking videos where characters speak, listen, and respond dyna…
MaineCoon: Pursuing A Real-Time Audio-Visual Social World Model
Lichen Bai, Tianhao Zhang, Shitong Shao +14
As an increasing majority of global video content is consumed on social platforms for interactive social purposes, video generation models built for social worlds are important but…
TurboTalk: Progressive Distillation for One-Step Audio-Driven Talking Avatar Generation
Xiangyu Liu, Feng Gao, Xiaomei Zhang +4
Existing audio-driven video digital human generation models rely on multi-step denoising, resulting in substantial computational overhead that severely limits their deployment in r…
AdaField: Generalizable Surface Pressure Modeling with Physics-Informed Pre-training and Flow-Conditioned Adaptation
Junhong Zou, Wei Qiu, Zhenxu Sun +3
The surface pressure field of transportation systems, including cars, trains, and aircraft, is critical for aerodynamic analysis and design. In recent years, deep neural networks h…
Pose-RFT: Enhancing MLLMs for 3D Pose Generation via Hybrid Action Reinforcement Fine-Tuning
Bao Li, Xiaomei Zhang, Miao Xu +3
Generating 3D human poses from multimodal inputs such as images or text requires models to capture both rich spatial and semantic correspondences. While pose-specific multimodal la…
Improving Large Vision-Language Models' Understanding for Flow Field Data
Xiaomei Zhang, Hanyu Zheng, Xiangyu Zhu +4
Large Vision-Language Models (LVLMs) have shown impressive capabilities across a range of tasks that integrate visual and textual understanding, such as image captioning and visual…
SyncTalk++: High-Fidelity and Efficient Synchronized Talking Heads Synthesis Using Gaussian Splatting
Ziqiao Peng, Wentao Hu, Junyuan Ma +7
Achieving high synchronization in the synthesis of realistic, speech-driven talking head videos presents a significant challenge. A lifelike talking head requires synchronized coor…
SyncTalk: The Devil is in the Synchronization for Talking Head Synthesis
Ziqiao Peng, Wentao Hu, Yue Shi +6
Achieving high synchronization in the synthesis of realistic, speech-driven talking head videos presents a significant challenge. Traditional Generative Adversarial Networks (GAN)…