7 papers
MirrorMe: Towards Realtime and High Fidelity Audio-Driven Halfbody Animation
Dechao Meng, Steven Xiao, Xindi Zhang +5
Audio-driven portrait animation, which synthesizes realistic videos from reference images using audio signals, faces significant challenges in real-time generation of high-fidelity…
Controllable and Expressive One-Shot Video Head Swapping
Chaonan Ji, Jinwei Qi, Peng Zhang +2
In this paper, we propose a novel diffusion-based multi-condition controllable framework for video head swapping, which seamlessly transplant a human head from a static image into…
OmniTalker: One-shot Real-time Text-Driven Talking Audio-Video Generation With Multimodal Style Mimicking
Zhongjian Wang, Peng Zhang, Jinwei Qi +5
Although significant progress has been made in audio-driven talking head generation, text-driven methods remain underexplored. In this work, we present OmniTalker, a unified framew…
Exploring Timeline Control for Facial Motion Generation
Yifeng Ma, Jinwei Qi, Chaonan Ji +4
This paper introduces a new control signal for facial motion generation: timeline control. Compared to audio and text signals, timelines provide more fine-grained control, such as…
ChatAnyone: Stylized Real-time Portrait Video Generation with Hierarchical Motion Diffusion Model
Jinwei Qi, Chaonan Ji, Sheng Xu +3
Real-time interactive video-chat portraits have been increasingly recognized as the future trend, particularly due to the remarkable progress made in text and voice chat technologi…
Animate Anyone 2: High-Fidelity Character Image Animation with Environment Affordance
Li Hu, Guangyuan Wang, Zhen Shen +6
Recent character image animation methods based on diffusion models, such as Animate Anyone, have made significant progress in generating consistent and generalizable character anim…