8 citations · 9 across the 14 of their papers we have counts for
15 papers · 1 filter
AVC-Reward: Event-Centric Audio-Video Consistency Evaluation
Yuruo Wu, Kaihang Pan, Kai Shen +6
As joint audio-video generation models become more capable, they can generate audio-visual scenes exhibiting three forms of complexity: multi-event composition, multi-layer audio m…
Prism: Dynamic Sparse Attention for Native 2K Joint Video-Audio Generation Model Training
Shuyuan Tu, Qi Tian, Yinming Huang +8
Natively training joint video-audio generation models at higher resolutions empowers them to learn richer visual details and sharper motion dynamics. However, full attention incurs…
VA-Judger: Reward Modeling from Human Preference Feedback for Joint Video-Audio Generation
Yinming Huang, Shuyuan Tu, Xi Yan +6
Using reinforcement learning to post-train joint video-audio generation models requires a reward signal. Existing methods construct this reward by combining metrics for individual…
Agentic Visual Generation: From Generative Models to Agentic Control
Yinming Huang, Shuyuan Tu, Xi Yan +7
Visual generation is evolving from generative models used through a single invocation into agentic control processes that can plan, select tools, inspect intermediate synthesized o…
Baton: Explicit Semantic Blueprints for Joint Video-Audio Generation
Shuyuan Tu, Qi Tian, Zihan Yang +9
Current open-source diffusion models struggle to generate stable and synchronized audio-visual content, particularly in scenarios demanding complex semantic reasoning. The root cau…
Preference Score Distillation: Leveraging 2D Rewards to Align Text-to-3D Generation with Human Preference
Jiaqi Leng, Shuyuan Tu, Haidong Cao +4
Human preference alignment presents a critical yet underexplored challenge for diffusion models in text-to-3D generation. Existing solutions typically require task-specific fine-tu…
Emotion-LLaMAv2 and MMEVerse: A New Framework and Benchmark for Multimodal Emotion Understanding
Xiaojiang Peng, Jingyi Chen, Zebang Cheng +11
Understanding human emotions from multimodal signals poses a significant challenge in affective computing and human-robot interaction. While multimodal large language models (MLLMs…
ArcFlow: Unleashing 2-Step Text-to-Image Generation via High-Precision Non-Linear Flow Distillation
Zihan Yang, Shuyuan Tu, Licheng Zhang +3
Diffusion models have achieved remarkable generation quality, but they suffer from significant inference cost due to their reliance on multiple sequential denoising steps, motivati…
FlashPortrait: 6x Faster Infinite Portrait Animation with Adaptive Latent Prediction
Shuyuan Tu, Yueming Pan, Yinming Huang +6
Current diffusion-based acceleration methods for long-portrait animation struggle to ensure identity (ID) consistency. This paper presents FlashPortrait, an end-to-end video diffus…
StableAvatar: Infinite-Length Audio-Driven Avatar Video Generation
Shuyuan Tu, Yueming Pan, Yinming Huang +6
Current diffusion models for audio-driven avatar video generation struggle to synthesize long videos with natural audio synchronization and identity consistency. This paper present…
PersonaAnimator: Personalized Motion Transfer from Unconstrained Videos
Ziyun Qian, Runyu Xiao, Shuyuan Tu +7
Recent advances in motion generation show remarkable progress. However, several limitations remain: (1) Existing pose-guided character motion transfer methods merely replicate moti…
StableAnimator++: Overcoming Pose Misalignment and Face Distortion for Human Image Animation
Shuyuan Tu, Zhen Xing, Xintong Han +5
Current diffusion models for human image animation often struggle to maintain identity (ID) consistency, especially when the reference image and driving video differ significantly…
StableAnimator: High-Quality Identity-Preserving Human Image Animation
Shuyuan Tu, Zhen Xing, Xintong Han +4
Current diffusion models for human image animation struggle to ensure identity (ID) consistency. This paper presents StableAnimator, the first end-to-end ID-preserving video diffus…
MotionFollower: Editing Video Motion via Lightweight Score-Guided Diffusion
Shuyuan Tu, Qi Dai, Zihao Zhang +6
Despite impressive advancements in diffusion-based video editing models in altering video attributes, there has been limited exploration into modifying motion information while pre…
MotionEditor: Editing Video Motion via Content-Aware Diffusion
Shuyuan Tu, Qi Dai, Zhi-Qi Cheng +4
Existing diffusion-based video editing models have made gorgeous advances for editing attributes of a source video over time but struggle to manipulate the motion information while…