11 citations · 16 across the 22 of their papers we have counts for
6 papers · 1 filter
AVC-Reward: Event-Centric Audio-Video Consistency Evaluation
Yuruo Wu, Kaihang Pan, Kai Shen +6
As joint audio-video generation models become more capable, they can generate audio-visual scenes exhibiting three forms of complexity: multi-event composition, multi-layer audio m…
Prism: Dynamic Sparse Attention for Native 2K Joint Video-Audio Generation Model Training
Shuyuan Tu, Qi Tian, Yinming Huang +8
Natively training joint video-audio generation models at higher resolutions empowers them to learn richer visual details and sharper motion dynamics. However, full attention incurs…
VA-Judger: Reward Modeling from Human Preference Feedback for Joint Video-Audio Generation
Yinming Huang, Shuyuan Tu, Xi Yan +6
Using reinforcement learning to post-train joint video-audio generation models requires a reward signal. Existing methods construct this reward by combining metrics for individual…
Iron: Intent-Aligned and Retrospective Dual Learning Framework for Enhancing Generalist Virtual Agents
Jiahe Ying, Wendong Bu, Kaihang Pan +6
Achieving virtual agents capable of automating tasks across diverse digital environments remains a pivotal challenge in Embodied AI. While Multimodal Large Language Models (MLLMs)…
SpatialFusion: Endowing Unified Image Generation with Intrinsic 3D Geometric Awareness
Haiyi Qiu, Kaihang Pan, Jiacheng Li +3
Recent unified image generation models have achieved remarkable success by employing MLLMs for semantic understanding and diffusion backbones for image generation. However, these m…
OmniWeaving: Towards Unified Video Generation with Free-form Composition and Reasoning
Kaihang Pan, Qi Tian, Jianwei Zhang +12
While proprietary systems such as Seedance-2.0 have achieved remarkable success in omni-capable video generation, the academic research community lags far behind: most of its model…