9 citations · 9 across the 18 of their papers we have counts for
21 papers
Wan-Animate-2: Pushing the Application Boundaries of Character Animation
Guangyuan Wang, Li Hu, Dechao Meng +11
Character image animation remains a foundational yet challenging task in computer vision. Existing approaches can be broadly categorized into three paradigms: methods based on expl…
Real-Time Human-Centric World Modeling for Upper-Body Human-Object Interaction
Chaonan Ji, Jinwei Qi, Peng Zhang +1
We present a real-time human-centric world model for upper-body interactive generation, aiming to synthesize coherent local world dynamics centered on a person, where coordinated b…
Wan-Dancer: A Hierarchical Framework for Minute-scale Coherent Music-to-Dance Generation
Mingyang Huang, Peng Zhang, Li Hu +5
Generating long-duration, high-definition, and rhythmically synchronized dance videos directly from music remains a significant challenge, primarily due to the temporal constraints…
MultiAnimate: A Unified Framework for Controllable Multi-Character Animation
Zhongyi Zhang, Guangyuan Wang, Li Hu +6
Recent advances in generative models and technological innovations have significantly addressed the fundamental challenges of character image animation. However, existing approache…
WanToFight: Real-Time Generative Game Engine for Multi-Player Combat Interaction
Li Hu, Guangyuan Wang, Peng Zhang +1
We present WanToFight, a generative game engine that simulates real-time, two-player The King of Fighters '97 (KOF~'97) gameplay from keyboard input. Prior generative game engines…
PortraitDirector: A Hierarchical Disentanglement Framework for Controllable and Real-time Facial Reenactment
Chaonan Ji, Jinwei Qi, Sheng Xu +2
Existing facial reenactment methods struggle with a trade-off between expressiveness and fine-grained controllability. Holistic facial reenactment models often sacrifice granular c…
UCM: Unified Modeling of Camera Control and Memory with Time-aware Positional Encoding Warping for World Models
Tianxing Xu, Zixuan Wang, Guangyuan Wang +5
World models based on video generation demonstrate remarkable potential for simulating interactive environments yet suffer from persistent difficulties in two key areas: maintainin…
Knot Forcing: Taming Autoregressive Video Diffusion Models for Real-time Infinite Interactive Portrait Animation
Steven Xiao, Xindi Zhang, Dechao Meng +3
Real-time portrait animation is essential for interactive applications such as virtual assistants and live avatars, requiring high visual fidelity, temporal coherence, ultra-low la…
Co-speech Gesture Video Generation via Motion-Based Graph Retrieval
Yafei Song, Peng Zhang, Bang Zhang
Synthesizing synchronized and natural co-speech gesture videos remains a formidable challenge. Recent approaches have leveraged motion graphs to harness the potential of existing v…
SyncAnyone: Implicit Disentanglement via Progressive Self-Correction for Lip-Syncing in the wild
Xindi Zhang, Dechao Meng, Steven Xiao +3
High-quality AI-powered video dubbing demands precise audio-lip synchronization, high-fidelity visual generation, and faithful preservation of identity and background. Most existin…
MirrorMe: Towards Realtime and High Fidelity Audio-Driven Halfbody Animation
Dechao Meng, Steven Xiao, Xindi Zhang +5
Audio-driven portrait animation, which synthesizes realistic videos from reference images using audio signals, faces significant challenges in real-time generation of high-fidelity…
Controllable and Expressive One-Shot Video Head Swapping
Chaonan Ji, Jinwei Qi, Peng Zhang +2
In this paper, we propose a novel diffusion-based multi-condition controllable framework for video head swapping, which seamlessly transplant a human head from a static image into…
MotionRAG-Diff: A Retrieval-Augmented Diffusion Framework for Long-Term Music-to-Dance Generation
Mingyang Huang, Peng Zhang, Bang Zhang
Generating long-term, coherent, and realistic music-conditioned dance sequences remains a challenging task in human motion synthesis. Existing approaches exhibit critical limitatio…
Exploring Timeline Control for Facial Motion Generation
Yifeng Ma, Jinwei Qi, Chaonan Ji +4
This paper introduces a new control signal for facial motion generation: timeline control. Compared to audio and text signals, timelines provide more fine-grained control, such as…
OmniTalker: One-shot Real-time Text-Driven Talking Audio-Video Generation With Multimodal Style Mimicking
Zhongjian Wang, Peng Zhang, Jinwei Qi +5
Although significant progress has been made in audio-driven talking head generation, text-driven methods remain underexplored. In this work, we present OmniTalker, a unified framew…
ChatAnyone: Stylized Real-time Portrait Video Generation with Hierarchical Motion Diffusion Model
Jinwei Qi, Chaonan Ji, Sheng Xu +3
Real-time interactive video-chat portraits have been increasingly recognized as the future trend, particularly due to the remarkable progress made in text and voice chat technologi…
Animate Anyone 2: High-Fidelity Character Image Animation with Environment Affordance
Li Hu, Guangyuan Wang, Zhen Shen +6
Recent character image animation methods based on diffusion models, such as Animate Anyone, have made significant progress in generating consistent and generalizable character anim…
DanceMeld: Unraveling Dance Phrases with Hierarchical Latent Codes for Music-to-Dance Synthesis
Xin Gao, Li Hu, Peng Zhang +2
In the realm of 3D digital human applications, music-to-dance presents a challenging task. Given the one-to-many relationship between music and dance, previous methods have been li…
VividTalk: One-Shot Audio-Driven Talking Head Generation Based on 3D Hybrid Prior
Xusen Sun, Longhao Zhang, Hao Zhu +7
Audio-driven talking head generation has drawn much attention in recent years, and many efforts have been made in lip-sync, expressive facial expressions, natural head pose generat…
MaTe3D: Mask-guided Text-based 3D-aware Portrait Editing
Kangneng Zhou, Daiheng Gao, Xuan Wang +9
3D-aware portrait editing has a wide range of applications in multiple fields. However, current approaches are limited due that they can only perform mask-guided or text-based edit…
Animate Anyone: Consistent and Controllable Image-to-Video Synthesis for Character Animation
Li Hu, Xin Gao, Peng Zhang +3
Character Animation aims to generating character videos from still images through driving signals. Currently, diffusion models have become the mainstream in visual generation resea…